Building to Scale with OCPP: Webinar with Pon Paulraj of Heliox

In eDRV's first webinar of the year, eDRV tackled the issue of how the Open Charge Point Protocol (OCPP) can help EV charging networks build to scale. R&D Manager Pon Paulraj of Heliox joined eDRV CTO and co-founder Rupul Safaya to talk about the challenges and intricacies of using OCPP as businesses roll out 1,000s of chargers at scale.

‍

What are the crucial first steps for a charging network when implementing OCPP?

Pon Paulraj: The first step is to start with the business requirements. This might sound like a very generic answer but this is relevant because OCPP is a basic communication between the charger and the CSMS. It doesn't specify too much about the business logic so it doesn't say what you can build over the OCPP data. That's why it's important to have a vision of what exactly you want to do with the OCPP data.

The business requirements of a private fleet operator could be a lot different from a public charge point operator. For a public charge point operator there are local regulations that decides what is the right hardware and what is the right software for your applications. For example, if you have to operate in Germany you have to be Eichrecht compatible where the hardware as well as software part of the OCPP should be ready to accommodate the Eichrecht updates.

My advice or experience is not to start coding immediately. First try to understand the business requirements with the OCPP data and then get into the implementation.

Rupul Safaya: “Don't start coding immediately” is something that doesn't just apply to OCPP. We're a startup and in general there's a principle of lean startups which is to really double down on your business requirement. 

Generally speaking, we must remember that OCPP is a communications protocol that defines how hardware charge stations should speak to the server side. At the end of the day it is establishing a messaging highway.

Drivers and cars don't really care about OCPP implementation. You could have the most efficient messaging backend but what drivers care about is to be able to walk up to a location, charge and drive away. So whenever we sit down with our customers we always start with this conversation about what the business differentiation is that they're looking for. 

If people are coming to OCPP anew and are looking to implement their own stack from scratch, the market is not filled with experts to pick up and start building. This so that's one of the first things that you’ll come across.

The second part of this is infrastructure, which includes websocket servers and messaging cues. You really need to think carefully about why infrastructure is so important to your business case. For most businesses that are looking to scale, the messaging stock is super important. It's like a utility. You need to rely on it but it's not the key differentiator.

‍

What sort of timescale can you expect in your first steps with OCPP? 

Safaya: How long will depend on your business requirement. If you're just doing a university project to prove that you can take messages from a charger and show them on a dashboard, that would be a shorter amount of time. But if you're heading towards scale then you really need to think about lining up a lot of different elements.

Paulraj: A reason why I said business requirements as a first point is because OCPP has seven profiles like core profile, firmware management, multiple things. Most of the players just implement the core profile but that has very limited functionality. 

If you know you’re going to deploy this in a private fleet then you know what parameters are crucial to focus on. Otherwise you might implement a core profile, go into the field and then come up with the big surprise later! 

So the duration to ramp up depends on what you want to implement and to what detail you want to implement it. It's important to have a clear vision on what exactly you want to achieve out of an OCPP implementation and then make a plan accordingly that decides its duration. It can be within three months you have a first MVP or it can take up to nine months.

What are the main challenges that you can address with OCPP as you scale to 1000s of chargers?

Safaya: An OCPP implementation in its early stages will look like picking up any skill like golf. Initially it's about “how difficult can this be?”. Then as you pick up any skill you really need to work harder. Over time you'll find that with your first implementation, the servers and infrastructure that you've set up is no longer enough. 

You need to start doing something called “horizontal scaling”, essentially adding servers. In that paradigm of multiple devices, where one machine starts breaking down, your engineering team is going to start looking at microservices. That will start introducing a level of complexity that wasn't there before. This is something that other web services come across. It's something that you have to take care of now.

The second challenge that is unique to OCPP is the strictness of the ordering in which messages should arrive to your application. In initial days you may get away with this because traffic's slow and there's no latency in your processing. But over time your downstream applications require ordered messages that, at scale, have alarms in place in case something goes out of whack. 

Finally, all of this is infrastructure. You're adding servers and your team's excited to add more servers. But on the cloud provider side your cost is going up. You're essentially doing the same thing where you're processing messages from left to right but it starts adding up. And don’t forget, as you're scaling from a software perspective, security and compliance becomes important very quickly too.

Paulraj: I’d like to explain the challenges of scaling more from the business side, things that I have seen at my previous job at Vattenfall. When I joined we were around 3,000 charge points and then at one point it was over 40,000 charge points. The scale up phase was exciting and everybody loves that period but it comes with a lot of growing pains.

First, as Rupul was saying, the huge load is unavoidable. When you are operating 2,000 chargers, a local server was sufficient and we had a server running perfectly fine with 100% uptime in Berlin. But when we were approaching 10,000 charge points that server was not enough anymore. Within two years we were expecting a growth of 30,000 charge points so we knew that we needed to move to the cloud. It's an easy decision to make but the implementation was super hard.

Moving from a local server was a big project because the old hardware, which was built around 2015, didn't support the automatic switching off of OCPP endpoint URLs. So we asked many of the hardware vendors to create a new firmware that can support this switching. They tried but my colleagues still had to drive to roughly 400 locations to plug in the land cable and then change the endpoint. Every large CPO who started their operation five or six years ago might have witnessed this.

While scaling up, tracking every important parameter is crucial. If you don't track or monitor anomalies it can cost you big time. This includes monitoring the data cost on the cloud and the number of messages on the SIM card. Typically charge stations don't consume so much SIM or GSM data but once I got an invoice from a telecom operator for €21,000 in a month! A buggy firmware which we didn’t validate completely went into the field which affected roughly around 300 charge points and they started sending a lot of garbage data into the back office. 

‍

What are some of the physical challenges in scaling?

Paulraj: When you are small there is two station installation in a day and everyone is happy as you can do everything manually. But when you are handling thousands of chargers with maybe 100 installations in one day, it’s a huge load for the operations team.

Working on bulk Device Management, bulk Configuration Management and bulk Firmware Management are what you see from the physical side. If you don't handle it with good infrastructure you end up doing a lot of manual work. At one point in time we were around 40,000 charge points in my previous company which meant at least 15-20 different firmware versions. So the product development teams cannot just develop new products anymore. They have to constantly look at different firmware versions, constantly validate them and then roll it out. 

Safaya: I can feel your pain because we're on the other side of that! I wanted to add that there's a silent participant in our industry that plays a very key role that isn't talked about enough, which is the role of the telecoms or internet provider. 

In the early stages we've had customers who will go and install a cable themselves or their first couple of sites will be in prime locations where they have a near line of sight to the cell towers.

But if your teams are doing like 20 to 30 wallboxes at a time, things like what the cell signal is like gets forgotten. For some of our customers we've had to teach them how to do wireless site surveys because, even if it's a DC charging installation, what can take some teams by surprise is the security and networking aspect even if you're doing a wired install. Either you have to take that into account by having teams that have that skill and are able to execute the telecoms part of EV charging.

‍

Watch the full webinar replay for more on using OCPP to scale. If you have more questions on anything you may have seen in this recap, feel free to reach out to us here.