Master’s student manages to make ASML spare parts supply chain 5 percent more efficient
How AI improves service network for Dutch chip machines.
For his thesis, Operations Management & Logistics and Data Science in Engineering master’s student Joost van der Haar did an internship at chip machine manufacturer ASML in Veldhoven. There, he used artificial intelligence to develop an improved model for the supply chain of spare parts for the advanced chip machines. According to Van der Haar, it could save the company millions of euros each year.
When an advanced chip machine rolls off the line at ASML in Veldhoven and is installed at a customer’s site, the idea is that it should keep on working – always. In the competitive world of chipmakers, downtime quickly translates into a loss of millions.
For this reason, ASML has a global network consisting of more than 60 distribution points with spare parts, which should ensure that a part can always reach the customer in time for maintenance. A chip machine has thousands of parts and, in the distribution network, tens of thousands of different spare parts are circulated.
All those components have their own average lifespan and delivery time.
Own logistics system
To manage the supply of components, ASML uses the system NORA (Network Oriented Replenishment Automation). This produces a new replenishment plan every day. It meticulously tracks how many parts are in stock, shifts between locations if necessary, and orders new components.
TU/e master’s student Joost van der Haar did a ten-month internship at ASML and took a close look at this replenishment system for spare parts. He saw that it could be improved. By training a deep reinforcement learning model (a form of artificial intelligence) with supply chain behavior, he created a new algorithm that he believes is more efficient than that of ASML.
The company could save about 5 percent each year on transportation to supply the thousands of machines worldwide, among other things. Machines would also be down for less time.
Artificial intelligence
Logistics is a balancing act. Of course, you want there to be enough spare parts for machines at all times; on the other hand, it is also inefficient to (unnecessarily) have a great deal of stock. ASML determines the optimum amount of tens of thousands of parts in its service network.
According to Van der Haar, the most efficient inventory depends on many different elements. If a part typically breaks down quickly, more stock is needed. It also depends on the type of service contract entered into between ASML and the customer. In addition, parts need to be in the right location.
“Most of ASML’s stock locations are logically close to the customers, such as the major chip manufacturers in Taiwan,” says Van der Haar.
Scale poses a challenge
Due to the scale of this distribution network, it is actually mathematically impossible to create a perfect distribution plan. The planning software NORA does make an attempt at this and typically finds a good solution.
Van der Haar tried to tackle the problem with artificial intelligence (AI), which is usually well-suited to this type of optimization problem. The DynaPlex algorithm (see text frame) he used was developed by Willem van Jaarsveld, an associate professor of Stochastic Optimization and Machine Learning at TU/e.
Van der Haar fed the AI model with information about ASML’s supply network. The data came from a simulation of the entire chain of spare parts. “Unlike ASML’s NORA model, I don’t tell DynaPlex how to behave. The AI determines those on its own, based on the data from the simulation,” says Van der Haar.
Training
After training, the artificial intelligence came up with its own replenishment plan. In doing so, it appeared to do a number of things differently and more efficiently than NORA. For example, the researcher saw that it regularly suggested movements of parts between two ‘local’ warehouses that are close to the customer.
“It may indeed be more efficient to replenish parts from local inventories rather than from a central inventory in Veldhoven,” says Van der Haar. “NORA does not yet make this kind of suggestion.”
He also saw something that he calls ‘ABCs’, whereby stock is replenished in a sort of train. An empty location C then gets parts from location B (which is nearby), which in turn gets stock from central distribution point A. “In terms of shortage risks, this can be more beneficial than replenishing all local sites directly from point A,” says Van der Haar.
Challenges
According to the master’s student, there are challenges in training the artificial intelligence. Firstly, it is difficult to incorporate multiple locations; the complexity of the computational problem then grows rapidly. In addition, it is notoriously difficult to apply deep reinforcement learning to events that do not occur often, such as ordering a part that only occasionally needs replacement.
“Think of the model as a dog that you want to teach a trick. If you don’t reward the dog with a treat often enough, for example, it won’t learn well. To get around this, I rewarded the model in a different way during training: on the basis of expected future demand,” says Van der Haar.
Saving millions on spare parts – why wouldn’t ASML implement this algorithm right away? Van der Haar thinks that the company will use his recommendations for short-term planning at least.
“They initially want to see if this will allow them to tighten the current replenishment rules,” he says. If this fails, the company will likely look at the DynaPlex model.
Paper and true reality
But even if artificial intelligence creates a better distribution plan, there will always be complicating factors. For example, there is a difference between paper and reality.
“There are many human links in the chain. For example, planners see that there are indeed parts in the warehouse but that they appear to be damaged. That can be a reason to have extra stock, just to be on the safe side,” says Van der Haar.
“Parts can also simply get lost or be in the wrong place in the warehouse.” The relevant stock information is spread across many people, locations, and systems; in that sense, many exceptions and also translations are needed.
ASML vs. Amazon
Other companies are now also looking at planning their stocks using artificial intelligence, such as Internet giant Amazon. Their researchers recently published a model that makes purchasing decisions for replenishing tens of thousands of products.
The number of products is very easy to scale up in this kind of model, says Van der Haar. “The crux is that such a model does not look at one specific product but at all products at once. In fact, it doesn’t matter whether there are a thousand or a million products.”
What the models do need to be adjusted for is the number of distribution locations. Amazon’s model considered just a single location. Other algorithms have considered up to 11 locations at a time but for one product.
Van der Haar’s research is unique because it is based on many different parts and 60 locations. Without the mathematical tricks he applied, the calculation problem for ASML’s supply chain would not have been solvable within a feasible timeframe.
Focus on efficiency
Supervisor Van Jaarsveld thinks Van der Haar has done a fantastic job. “Through his creativity and intelligence, he has made the most of using DynaPlex,” he says. That there was still room for improvements in ASML’s service network does not surprise him – logistical problems of this scale are “just really difficult”.
He says it is important for logistics to become even more efficient. It is a branch of business that uses a lot of resources. AI can help achieve these goals more efficiently. “Companies have to be convinced of its usefulness first, though, and that takes time. Given the progress in recent years, I expect more applications of this in the near future.”
What has stayed with Van der Haar from his time at ASML is the company’s focus on efficiency. “Almost everyone is improving processes every day,” he says. “There are extensive roadmaps, leading to the creation of more machines and better machines. Of course, every company is constantly trying to do better, but I got the impression that this is really the focus at ASML.”
Van der Haar himself has just moved to Leuven to pursue a PhD at the university on the use of artificial intelligence in logistics. Still, he thinks that he will eventually end up in the business world. The more practical problems there are exactly what attracts him.
Joost van der Haar graduated this fall with his master’s thesis ‘Industrializing Deep Reinforcement Learning for ASML’s Service Network’, which can be read here.
