Of late we have played much at Artusi:Cooking time.)
It all started with the idea to have at hand, on our smartphone, Artusi’s cooking recipes (he’s the one that canonicized the classical italian cuisine in the late XIX century). What we found instead was a nice little game we blayed everywhere: a Bejeweled clone with some nifty twists.
The gameplay is very rewarding and at the end of each recipe, there’s a bonus screen you have to solve with just one swap. If you manage to collect with this swap the given amount of ingredients you gain three gold leaves (used later in the game). Usually that means matching most if not all of the squares in the screen.
After a few trials, we soon discovered that some screens were hard to solve (and some even were bugged, no solution), or you could solve them spending the same gold leaves you were meant to gain from solving the level.
So we started writing a small python program that, given a textual representation of the level in form of a matrix, searched for all possible swap using a dumb brute-force algorithm and then applying all the reduction rules of the game (like deleting three-in-a-row, or whole lines if combo moves are made). This way we can find the optimal solution to the level. It’s a matter of a couple of hours.
But unfortunately it’s pretty unuser-friendly to write down a TXT file with an 8x8 representation of the screen, each time we come to the end of a recipe. We needed an automatic tool that would do this for us, starting with the screen capture of the level: enter OpenCV.
Our first trials were pretty dumb: first task was to understand among all the squares which one was the ingredient we needed to collect. OF course this one varies, can be rosemary, nutmeg, butter, carrots, whatever.
We tried going with exclusion: if we can recognize empty squares and squares containing cutlery, the remaining one are the ingredients we are looking for! WE tried logic, training, until we found that the most trusty was pattern matching, since we have only 4 types of cutlery in the game: we provided the sample ones for the algorithm.
next task was to correctly tell apart empty squares from our ingredients. This looked easy buy proved difficult since the empty squares are not empty, they have a wooden texture which can be mismatched for another square if we rely only on hue matching, luminosity etc.. in the end we first ruled out empty squares with a brute force lightness averaging technique, then we matched cutlery against our pattern and then, of course, what was left was our mysterious ingredient!
So we tried first to lighten up the image (with image recognition you almost always work on greyscale images) then we looked for the whitest squarese with a simple montecarlo method: it scatters random point on a square and if 80 out o 100 are “whitish”, it’s our background! We came to the threshold of 80 just after a few trials.
As we said before, pattern matching was not good because backdrops for empty squares vary.
When we had everything in place, we wrapped everything in a simple python web server, on aour rogue web site, and can be used from our smartphone, simply uploading the screenshot. After a few seconds the server will send back a grayscale image with higlighted the two squares to swap.
So when you are sitting upon your ivory thorne and absolutely need those golden leaves, it’s ready to use..