Blog: How Is The Role Of Technology Changing Optimisation?
Setting the Scene of Change
From an energy, and specifically electricity, perspective, what is optimisation? The purpose of optimisation varies depending upon the party leading this activity.
Within the power market of Great Britain (GB), the role of balancing electricity demand with supply at the point of delivery, and at least cost to the end consumer, is the responsibility of the National Energy System Operator (NESO). Optimisation in that sense is perhaps one of the most complex activities within the entire market, with many thousands of variables taken into consideration by the NESO, amongst which are system frequency, voltage, and reserve levels.
Complexity within the NESO balancing & optimisation process has risen considerably over the past decade and can be only expected to continue to increase over the next twenty years. Similarly, Asset Optimisers (AO) involved in optimising the output of generation assets in in the wholesale traded market (the supply) and those servicing the end user (demand) have faced similar levels of change.
The single biggest driver of this change has been provided by UK Government policy, which has targeted both the threat from climate change and the aim of increased competition on the customer supply-side.
So how have these policy decisions altered the electricity industry landscape?
Where Have We Come From?
Fig a.
As shown in Fig a, twenty years ago the country’s generation mix was heavily reliant upon fossil fuel, with perhaps 40-50 large Coal & Gas power stations around the country (alongside Nuclear), owned & traded by between 8-10 optimisers. Meanwhile the customer supply side was dominated by 6-7 large outfits, with minimal interaction between customers and the wholesale or balancing markets.
The role of NESO was simpler, with the options available to them for managing and balancing the system often covered by between 10-20 of these large power stations, and with NESO Control Room staff able to build & send manual dispatch instructions to these generators to balance & manage the transmission system via the Balancing Mechanism (BM).
Similarly, AOs also faced an easier task of scheduling their generation assets to maximise their value. With fewer, larger assets, Traders were able to set asset target output levels manually using relatively low specification IT software and were responsible for choosing whether to dispatch between either the wholesale market or the BM. Market rules were also less sophisticated, with minimal oversight of asset & trader behaviour as they interacted with these two routes-to-market.
Where Are We Now & Where Will Be?
Fig b represents the GB power market as it appears today, with the fleet of Coal and older Gas generators supplanted by hundreds of new wind generators, lower capacity gas reciprocating engines and battery storage sites, and thousands of small-sized solar PV installations, with many of these generation assets now located within the distribution network or directly connected to customer properties. This fundamental shift in the way electricity is generated across the country, from a few large generators to many, many smaller assets, has had profound implications for both how the NESO must balance & manage the transmission system and how AOs now optimise their portfolios.
The approach where NESO Control Room Engineers or AO Traders can manually instruct, or schedule, generators to balance and / or maximise value is no longer an effective or sustainable option. In addition, the ability of the BM to provide NESO with all the tools required to manage the system has waned, as rising levels of intermittent generation has necessitated the development of new & innovative ancillary balancing services that new generation technology types can provide.
Indeed, the next ten years is likely to see further structural changes, with an even greater amount of consumer involvement in the process of balancing supply with demand. The latter will be a crucial element of the nation’s transition to a net zero carbon electricity system, where greater amounts of intermittent renewable generation will require greater amounts of flexibility both in how electricity is produced and consumed.
Where Does Technology Feature Amongst This Change?
As was described above, the drive to net zero has & will continue to increase the complexity involved within the electricity market. Balancing supply with demand will require instructions to hundreds of generators rather than a handful, and managing the system will require many different services rather than two or three.
Technology, either through the need for AOs to tender for new balancing services via online auctions, or for their assets to be optimised & dispatched via algorithm, has thus played a crucial role in the net zero transition.
The procurement process for many of the NESO ancillary balancing services take place via Day-ahead auctions, that is the day before actual delivery begins. In addition, wholesale trading can also take place via Day-ahead auctions and as Within-day delivery arrives further wholesale trading via Exchanges is possible, as is utilisation via the BM.
Fig c.
Asset Optimisers must decide which of these many different options will deliver the best return for each of the assets within their portfolio. This is an extremely complex calculation with many input variables, hence the requirement for an algorithm that can identify the best outcome at the Day-ahead stage. This requires a range of forecast and static data inputs to be provided to the optimisation algorithm, with the Trader continuing to play a key role in the process, for instance by making assessments about which routes-to-market or services may be saturated / over-supplied given the NESOs requirements and recent competing asset optimiser behaviour.
The Day-ahead optimiser itself requires continuous improvement by a team of skilled subject matter experts (SMEs) & Software Engineers (SEs), enabling it to remain fit for purpose as new ancillary services are rolled-out by the NESO, or as new combinations of services or delivery strategies are identified.
Similarly, once the optimisation strategy at the Day-ahead stage has been determined and executed, Traders must then continue to feed data inputs into a distinct optimiser tailored to deliver contracted services and maximise all other opportunities presented, be them via wholesale markets or BM.
Fig d.
The Trader is responsible for forecasting market & BM prices across the Within-day horizon (shown in Fig d. above), passing these views in to the Within-day optimiser so that it may schedule the most economic dispatches to monetise the value of each asset.
The act of forecasting prices also requires a suite of supporting graphical user interface tools (see Fig e. below) that provide the Trader with the latest key market information from a range of sources, some of which can feed automatically into the optimisation process, and others which require monitoring and curation by the Trader to account for anomalous events.
Fig e.
Therefore, the growth of technology within the optimisation process has transformed the role of the Trader from a person that manually dispatches and trades assets into a focal point of market expertise, who can use semi-automated processes to feed data into algorithms that will now instead identify and create optimal dispatches and trading opportunities.
The NESO has seen similar technological changes impact the role performed by its Control Room Balancing Engineers, as well as the options available to them when managing the transmission system. Consequently, it has recognised within the past few years the need to evolve as an organisation with respect to how it utilises technology and is now in the middle of delivering a large programme of change (see Fig f. below).
Fig f.
By the end of 2025 the NESO will have originated six new balancing ancillary services, migrated all its daily auctions to a new platform that facilitates tendering via API, and adopted new automated processes within its Control Room that enable simultaneous dispatch of hundreds of assets via an algorithm that identifies the least cost balancing option (called the Open Balancing Platform – OBP).
Looking even further ahead, by 2035 could see greater technological integration, with consumers able to specify to their supplier or AO the amount of flexibility that their home could provide via their smart devices, domestic battery, or electric vehicles. Both the AO and the ESO could have access to hundreds of thousands of flexible assets, all capable of altering their output to balance the system or delivery ancillary services. None of this will be possible without greater reliance upon technological advances and increased automation.
What Does This Mean For The Role Of The Optimiser?
As has been highlighted so far, net zero, greater quantities of smaller assets, and many more balancing services and revenue streams for these assets, has required a profound change in the way that AO must operate and emphasised their dependency upon technology to assist in the optimisation process.
But what was only lightly touched upon was the value added by the employees at each AO to the optimisation process, whereby greater reliance on technology is only part of the solution. Rising complexity requires a deeper level of domain knowledge to be developed by the SMEs at the AO, who must combine understanding of balancing service delivery rules with awareness of the capabilities & limitations of their assets, and thus specifying to the SEs the parameters which new algorithms must operate within and what the target optimal outcomes should be.
In addition to SMEs and SEs, Data Scientists have a key role to perform, as the quality and breadth of data available to the market grows, creating the opportunity to improve the data inputs provided to the optimisation algorithms. Indeed, Data Scientists that have backgrounds as both SMEs and SEs will become a highly prized commodity within the industry.
Therefore, the AO must commit to invest heavily within its people, providing the time, space & support to nurture their development into tomorrow’s SMEs and Data Scientists, rewarding them sufficiently as their knowledge & skills grow.
The future goal of a net zero energy system will rely heavily upon the relationship between technology and human ingenuity.
Find out more about EDF Asset Optimisation Solutions: Asset optimisation solutions
Related articles
The power of co-location: Accelerating an integrated energy system
How NESO Keeps Electricity Supply Stable Using Balancing Services, Part 1
