But just the other day, while I was working on a report at my desk, it suddenly dawned on me – how on earth am I going to transport all my belongings to a new home? It should be said, I am a bit of a hoarder, and there is an entire wardrobe of art supplies that my parents are keen for me to take with me when I leave. Add to this my excess of fiction books, cuddly toy collection, and piano, and well… you’ve got a lot of things to transport 15 minutes down the road (you heard that right, I am moving a 15 minute drive away from my childhood home).

It turns out that packing problems actually constitute a well-studied area of optimisation. Often, in places like warehouses, the ability to effectively pack items of various sizes into a finite number of containers is vital to the smooth running of operations. Warehouse packing algorithms attempt to solve these kind of problems, by optimising the packing of items in day to day operations.

So what kind of packing problem is this?

The ‘bin packing problem’ is perhaps the most relevant description of my ‘dear lord how will I transport my cuddly toy collection‘ issue. Essentially, it refers to the problem of how best to pack multiple items of various sizes into a finite number of bins (or containers). Like most optimisation problems, it’s all about trying to find the best possible solution out of all the feasible solutions.

What do you think would give the best solution?

I think most people, when asked for the most optimal way to pack all my cuddly toys up, would suggest trying to fit as many cuddly toys as I can into one bin. And this would be a very good suggestion! This is what I love about operational research, you don’t need to know lots of fancy mathematical theory in order to have a good idea about how to solve a problem.

Generally speaking, the best solution for a bin packing problem can be one of two things:

1. Packing one container as densely as possible
2. Packing all objects into the least amount of packages as possible

Both of these solutions have very clear objectives: minimising cost. In some shipping companies, it is common that items will get charged for the amount of space they take up, rather than the weight. Hence, packing a container very densely is the best option.

Also, clearly reducing the amount of packages used reduces packaging costs.

Computational Complexity

if you’ve come across some form of optimisation problem before, you may have heard the term ‘computational complexity’ being thrown about. The computational complexity of a problem may sound scary, but in actual fact it can just be thought of as how many resources are required to solve the problem (maybe the resources are the time taken, or the data storage required).

When you measure how long a program takes to run when it is given more and more difficult problems (for example, packing 10 cuddly toys, then 20 cuddly toys, then 30 cuddly toys etc) you can plot the times and come up with a function.

In the case that the time taken increases exponentially or factorially (or anything that exceeds what a polynomial can do!) as the difficulty of the problem increases, we say that the problem is not solvable in polynomial time.

The bin packing problem is known as an NP-complete problem. This stands for ‘non-deterministic polynomial time’. This means that it is not solvable in polynomial time. In simplest terms, this kind of just means that if we increase the number of items that need to be packed, even by a little amount, then the problem takes wayyyy longer to solve.

A cool fact about NP-complete problems, is that if anyone could ever solve an NP-Complete problem in polynomial time, then we could solve all NP-complete problems in that way using the same method. This would mean that the entire class of NP-complete problems would cease to exist!

Standard Bin Packing Algorithms

So what are the different types of bin packing algorithms that exist?

Next fit: For this bin packing algorithm, I check if an item fits into the box that I am currently filling. If it does fit – great! I will add it to the box. If it doesn’t, I seal up the current box, ready to be transported to the new house, and I begin filling the next one. A benefit of this algorithm is that I can send the sealed boxes over to the new house one after another, so I am never required to pack more than one box at the same time – this helps save bedroom floor space while I pack!

First fit: For this algorithm, let’s imagine I have four boxes lined up on the floor, ready to be packed. Here, I take an item and I see if it will fit in the first box. If I does, I place it in, whereas if it doesn’t, I move onto the next box. I continue this down the row of four boxes, until I find one that works.

Worst fit: Imagine now that I have partially filled several boxes with items. For this algorithm, I will place my current item into the box with the least amount of items (as though I am trying to even the weight between all the boxes). If I have two equally empty boxes, I’ll put the item in the box that I started to fill first.

The above algorithms are known as online algorithms; they are used when items arrive one at a time, in an unknown order. In these algorithms, we must put the current item into a container before we consider the next item. Each algorithm has an associated runtime complexity, describing how the time taken for the algorithm to run scales with the number of items to be packed n. Ideally, we prefer algorithms with a lower runtime complexity, because this means that the algorithm performs quicker if it has to pack lots of items! But remember, we also want algorithms that give us good approximations to optimal solutions, so which you use depends on the situation.

Other algorithms exist for offline situations, where we have all the items available upfront (like my moving house case!). I won’t cover some of these here, but check out the further reading if you’d like to know more.

So there we have it!

I hope you enjoyed this small introduction to optimisation and bin packing algorithms. In reality, I may not use a bin packing algorithm to help pack my belongings in advance of moving day, but in the world of warehouse management, bin packing algorithms are really important. Many companies use 3D bin packing software to help optimise their operations.

There are several variations on general bin packing problems; for example, the classic optimisation problem known as the Knapsack problem. In the Knapsack problem, we have one bin, and our items are characterised by a value and a weight. Our goal is to maximise the value of the items that can fit into the bin.

If you did enjoy this post, and would like to read more, I’ve listed some great resources for further reading below!

• on the bin packing problem gives a great introduction, and goes into some more detail on the algorithms.
• If you’re interested in actually implementing some of these algorithms, I love , which describes how to code up all the algorithms discussed in this report in several different languages.
• is a great, easy to read description of how bin packing algorithms help warehouse operations.

This also extends into my academic career; you may have read on my home page that my undergraduate degree was in Theoretical Physics, but now I study Statistics and Operational research (check out this post about why I made the switch). Basically, I love to learn, and I love to discover as much as I can about the things I find interesting; hence my love of research.

One thing that has remained a constant love throughout my life though is art. I love to paint (in water colours, acrylics, dyes and oils – I told you, I enjoy everything), and also draw. I recently read a blog post by my friend Lídia, on the role of Operational Research in music – a truly interesting post that you can find here – and it got me thinking, I wonder if there was a link between Operational Research and art too?

Optimisation

Optimisation is a branch of Operational Research which considers the task of achieving optimal performance subject to certain constraints. Mathematically, optimisation problems deal with maximising or minimising some function with respective to some set; for example, given a set of decision variables x = (x₁, x₂, x₃, …, x_n), we want to find the decision variable that maximises or minimises the objective function.

The objective function could be the waiting time for customers in a system, or the profit from the sale of a product.

The subset which represents the allowable choices of decision variables is called the feasible set. Constraints in an optimisation problem work to specify what the feasible set is. For example, if we consider the waiting time for customers in a system, a constraint would be given by the fact the number of customers in a system can never be negative. This is a simple example of a constraint that one would encounter in an optimisation problem, but in reality there are often many constraints, that can be complex and highly dimensional. 

So how does this relate to art?

Well, clearly when an artist works, they are subject to some real-world constraints. They are required to work within budgets, meet deadlines, and follow the instructions of the customer in the case of commissions.

But, perhaps more interestingly, some artists subject themselves to constraints voluntarily. A Sunday on La Grande Jatte (1884) is a work by the French artist Georges-Pierre Seurat (see on ). Looking at it, the painting depicts Parisians having what looks to be a lovely day at the park. When viewed up close, however, it is possible to see that the painting is made up of many tiny dots of multicoloured paint. 

If we think about the creation of this painting as an optimisation problem, we could say that Seurat’s objective was to create the best possible depiction of what he saw, subject to two key constraints: only applying the paint in tiny dots, and keeping his colours separate. 

I came across , which discusses applying optimisation algorithms in order to create computer generated artwork. This general idea is considered for three different branches of art: line art, tile-based methods, and freeform arrangement of elements. For this post, we’re going to be considering tile-based methods, or mosaics. See .

When photographs or other images are represented digitally, we can think of this as a function which maps each pixel location (x, y) to some colour space. This is easy to consider in a mathematical context, for colours can be denoted using an RGB colour model or similar. Once photographs are represented in this form, it is easy for artists to construct a new artistic version of this photograph by replacing each pixel or block of pixels with some object, known as tiles.

The need for optimisation becomes apparent when an artist is choosing from a finite selection of tiles (where tiles could be anything ranging from dominoes, lego pieces, or other photographs). If the artist only has access to a fixed selection of tiles, then the goal is to find the ‘best’ overall arrangement of this set of tiles, in order to produce the most aesthetically pleasing artwork. Mathematically, the most ‘aesthetically pleasing’ artwork may be considered as the artwork which is most similar to the original image.

If we let I be a W x H image, and suppose that we have an inventory of tiles T = {T₁, T₂, …, T_n}. So how do we quantify how similar an approximation is to the original reference image? In the paper, a distance function d(I(x,y),T) is introduced, where I(x, y) is the colour of the pixel at location (x, y) in the original image, and T_j is the colour of a particular tile. This distance function provides a quantitative method for determining how effectively a given tile approximates a pixel in the source image.

We assume that d(I(x, y),T) is never negative, and smaller values denote better approximations (a distance value of zero would indicate that the tile and the corresponding source pixel are the same colour). If we consider all possible ways in which our inventory of tiles may be arranged on our image (where each tile is used no more than once), that is, all possible mappings, we then seek to find the mapping which minimises the total of the distance functions for every pixel.

In the case that our tiles match up with the dimensions of the pixels exactly, this proves to be a relatively simple task. It is possible to solve this by constructing a complete, weighted bipartite graph, in which each pixel location (x, y) is connected to every tile T_j by an edge of weight d(I(x, y),T_j). The minimum weight matching which uses all pixels can then be computed to give the optimal solution.

This task becomes far more complex in the case that the tiles do not match up one to one with the image pixels, for example, if the artist were to use something like dominoes. However, it is presented in the paper, that the construction of domino art can be naturally reduced to an problem.

Isn’t it fascinating how optimisation techniques can be used to create impressive artwork? I certainly think so. Some people might think that applying mathematical techniques such as integer programming to create art takes away from the spontaneity and skill that goes into creating a masterpiece, but I actually think it only adds to the awe! What do you think? Let me know in the comments.

Make sure to check out the further reading if you are interested in finding out more!

Further Reading

– Craig S. Kaplan, Robert Bosch

Ironically enough, learning from your mistakes, or past behaviour, is an idea that is strongly rooted in the multi-armed bandit problem. And thus, a blog post was inspired! 

Multi-armed bandits

So, let’s get started. What exactly is a multi-armed bandit?

When most people hear ‘multi-armed bandit’, they may think of gambling. That is because a one-armed bandit is a machine where you can pull a lever, with the hopes of winning a prize. But of course, you may not win anything at all. It is this idea which constitutes a multi-armed bandit problem.

Mutli-armed bandit problems are a class of problems where we can associate a particular score with each of our decisions at each point in time. This score includes the immediate benefit or cost of making that decision, plus some future benefit. 

Imagine that we have K independent one-armed bandits, and we get to play one of these bandits at each time t = 0, 1, 2, 3, …. These are very simple bandits, where we either win or lose upon pulling the arm. We’ll define losing as simply winning nothing.

Now, if your win occurs at some time t, then you gain some discounted reward a^t, where 0 < a < 1. Clearly, rewards are discounted over time; this means that a reward in the future is worth less to you than a reward now. The mathematically minded among you may realise that this means that the total reward we could possibly earn is bounded.

The probability of a success upon pulling bandit i is unknown, and denoted by π_i. Since these success probabilities are unknown, we have to learn about what the π_is are and profit from them as we go. This means that, in the early stages we have to pull some of the arms just to see what π_i might be like.

At each time t, our choice of which bandit to play is informed by each bandit’s record of successes and failures to date. For example, if I know that bandit 2 has given me a success every time I pulled it, I might be inclined to pull bandit 2 again. On the other hand, if bandit 4 has given me a failure most of the times, I might want to avoid this bandit. Thus, we are using previous data which we have obtained about each of the bandits in order to update our beliefs about the bandits’ probability distributions. 

The Maths

Updating our beliefs about the probability distributions of the bandits in this way is using an interpretation of statistics called Bayesian. Let’s imagine that we have a parameter that we want to determine (in our case, the probability of success for each of the K bandits). Maybe we have some prior ideas about what the probability of success for each bandit will be. This could be due to previous experiments you know about, or maybe just personal beliefs. These prior beliefs are described by the prior distributions for the parameters. 

Then, let’s imagine we begin our bandit experiment. After time t, we have pulled t bandits, and we now have information detailing the number of successes and failures for each of the bandits pulled. At this time, we take this observed data into account in order to modify what we think the probability distributions for the parameters look like. These updated distributions are called the posterior distributions.

The Question…

How do we play the one-armed bandits in order to maximise the total gain? 

Well, this is where the importance of learning from your mistakes comes in. Imagine the case where K = 5, and at time t = 7 our observed data are as follows:

Looking at our observed data, bandit 1 has the highest proportion of successes out of the number of times it was played. So maybe we would want to pull Bandit 1 again at our next time?

Well, maybe not.

Notice that Bandits 2 and 5 haven’t been played yet at all; therefore we can’t really infer any data about them. Pulling Bandit 1 might give us a success on our next attempt, but it could also give us a fail. We know nothing about Bandits 2 and 5, perhaps these bandits have a probability of success of 1? 

The idea of pulling the best bandits to maximise your expected number of successes relies on a balance between exploration and exploitation. Exploration refers to playing bandits that we don’t know much about, in this case, pulling arms 2 and 5. Exploitation, on the other hand, means pulling the arms of bandits that we already know might give us a good result; in our case, pulling bandit 1 again. 

Every time we pull an arm, we are actually receiving both a present benefit and a future benefit. The present benefit is what we win, and the future benefit is what we learn. Recall that at time t, if we win, we receive a discounted reward of a^t. Therefore, the closer a is to 1 determines how important it is to learn about the future. For example, if a is closer to one, many future pulls will make a big difference. On the other hand, if a is closer to 0, our present benefit is more important. Again, this relates to the importance of balancing exploration and exploitation.

So how do we quantify both the immediate benefit and the future benefit from these bandits, in such a way so that we can play the bandits to maximise our total gain? It is possible to take the posterior distribution for each bandit at each point in time, and associate a score with it that represents both future and present benefit of pulling that bandit. 

It turns out that there is something called a Gittins Index, which is a value that can be assigned to every bandit. By looking at the Gittins Index for all K bandits at time t, it is Bayes optimal to play any bandit which has the largest Gittins index, which is pretty cool!

So there we have it, learning from past experiences is critical in the world of multi-armed bandits, and also in many other areas of mathematics including reinforcement learning and bayesian optimisation. The trade off between exploration and exploitation is critical, and perhaps even serves as a reminder to the importance of not being afraid to make mistakes in our everyday lives… just a thought.

If you enjoyed this post, be sure to check out the following further reading resources!

Further Reading

This book gives a good introduction to multi-armed bandits for those interested, and goes into detail on some of their many uses.

are really great for those of you looking for a gentle introduction to the Gittins index and multi-armed bandits! Those new to the subject may also want to look at as a good starting point.

If you’re interested in exploration vs exploitation in reinforcement learning, and .

If you’re interested in exploration vs exploitation in Bayesian optimisation, .