Artistic machine 42

This is an introductory article about a social artificial intelligence artistmachine42 I am developing in my spare time and it will not contain a lot of math (unfortunately). More in depth articles will follow.

This is it’s most successfull post so far:

It seems I took a very long pause from writing on this site. I got on with a longer project I had in mind for a while and dedicated most of my time to accumulating knowledge. As I did not write for a while, this article will probably be kind of long and picture/tumblr post intensive :).

You can probably see, one of my passions is artificial intelligence. It’s been a great interest of mine since I first read an article my father wrote (when I was very little) for a science magazine which was about how people create computers basically by mimicking their way of perceiving the world, and trying to recreate it. The idea of the article (which I now, may admit that I don’t remember word for word) was actually a particularity of the idea that creativity and it’s produce is mostly constrained combinatorics and nothing fundamentally new can be created by a human being ( nice article about the general idea ) . You could see a big machine combining notions into ideas and then veto-ing the silly ones and could probably relate to the free won’t or neuroscience of free will but I did not know about these things then.

I really liked the concept at that time and it stuck to me. As I grew older, learned and tested things for myself, I became less and less sure of this. Actually I grew less and less sure about a lot of things but I guess that is the beauty of life, not knowing but actually finding out. Living in the constant state of ambivalence that keeps you creating and asking questions.

Artificial intelligence was always about “Oh my god, I think this is how I could model a human!” and later on “Guess I was wrong, back to the drawing board” and then “Oh my god, I think this how we model speech recognition/object recognition/strategic thinking!” and yet again “Guess I was wrong, back to the drawing board!”. This never-ending loop is not futile as this represents the scientific method of any domain and it got us really far. Every “guess I was wrong” statement would actually bring us closer to the truth.

Before talking about my project I will first like to express my awe about the google deep mind wining the Go contest (I had a lot of emotions watching the games unfold). Basically a part (the board analysis) of the Google AI was done with convolution neural networks (or so I read) so I guess it relates to my project a little, as I used the same idea for my style transfer stuff (the great article about this).

I followed some courses about neural networks but I would actually like to share with any interested reader that wants to know more about machine learning this free youtube course i really like that my friend and colleague Alexandru Ene brought to my attention.

My project is called ArtisticMachine42 and it’s actually a tumblr bot that posts neural network style transfer art and somewhat matching quotes, trying to basically create an experience in the viewer, the same way a human artist does.

This bot made me think of a lot of interesting questions about human relation to art, does the creator really matter? Maybe the creator matters only in his eyes, and all art actually happens in the viewer or maybe art is a bridge between people, like Whitmans spider silk (and machines can never take part in this). There are a lot of good questions one can ask with this experiment and because I didn’t want to spoil the informational part and be open to new discovery I didn’t write about my bot earlier (wanted to see if people saw the machine part or the art part). Maybe it relates to the turing test or it was simply a curiosity I had after playing around the bot or not site and creating a ton of bad poem generators and my licence paper that was also about creating a believable AI.

What I can say for sure is that my bot managed to get around 620 followers (although some of them porn bots :D ) at 522 posts spread out about 2 per day some people contacted it and wrote they really enjoyed his posts (around 20) some reblogs, an the most liked post had about 192 notes. I did not try to get followers, and tried to not delete anything it posted (the act of filtering its creations would actually imply a process that the AI is not yet capable of).. I actually deleted about 3-4 posts that were created with gruesome porn pics (before I decoupled the social part from the artistic part).

Even with me knowing it was a bot, and developing the algorithms, I still liked some of its posts. Sometimes artisticmachine42 seemed to post things I was thinking about :D . For me it was not magic, or synchronicity it was just my mind trying to find contextual patterns in it’s never ending problem solving, like a big fuzzy pattern contextual recognition device but nevertheless, i still enjoyed some of the posts even knowing they were not made by a human.

The process behind this is fairly complicated as I kept doing iterations (which can actually be seen through the posts as the bot gets better).

##The first iteration was just filter-combining the pictures it got from tumblr friends and then posting poetry generated by an n-gram statistical system trained on Shakespeare and Ovid (the results were kind of bad).

and a random test for procedural pencil painting that ended up being the logo

##The second iteration was doing a really expensive neural iterative process of painting and learning on each try by looking at several classical artist paintings this turned out to be too much for my poor server (dual core with no graphics card at that time) so I had to leave this thought behind.

• first steps

• more steps

  http://artistmachine42.tumblr.com/post/126477360347

• even more steps

##The third iteration was generating really short gifs and posting them via moviepy, from various movies it got from the internet and then it also posted a n-gram poem.

you can see here a lucky match :) :

##The forth iteration was actually the basics for what I have right now (I am just improving this via various methods and not changing the whole concept). We have 2 parts that work independently : the social part and the artist part. But I will first show you some results I liked:

• The social part: Handles the befriending, following and liking of tumblr accounts/posts
• The artist part: Handles the neural style transfer, finding good art styles and good photos to repaint and of course matching the quotes to the painting.

The social part is the easiest as it’s using the tumblr api to get friends of friends or followers and follow them randomly and also like posts.pytumblr

I basically have a sqlite database that started out with some hand picked accounts (artistically inclined, that I liked) and I get random friends of these guys, follow and add them to my database. (it’s crawling for friends on the internet :) ). I actually used friends pictures at first, for the artistic part but in the end this turned out not to be that great as the friends of friends of friend of… etc eventually turned out to be porn blogs and if you have just one porn blog friend then you get into the well connected graph of porn blogs and there is no exiting this :) , so I decoupled the artistic part from the social part (you may see this in older posts :) ).

The artistic part was a lot of fun to get working. I actually used caffe after trying tensor flow and another library I didn’t like as I had a google pre-trained model. I managed to make caffe work on windows (that was a real challenge) so I could develop in windows and deploy on Linux (my server). I optimized a lot so I got from a picture taking 8 hours on my server (quad core Xenon 2.3 GH, 10 gigs of CRC server Ram) and sometimes giving me OOM, to it taking 4 hours on my server (and 30 minutes on my personal computer with a CUDA enabled NVIDIA GEForce 970). Testing things out, making a table for the styles and pictures and programming this was a blast. Then I crawled for famous quotes (stored in my sqlite db, which I used for everything even dev settings). I loved working on this as I kept having nice questions that needed answering. The last small iteration was on quote matching with the paintings, after trying a lot of options I now use the google vision service to tag the art and then use these words to match a good quote. I have a good idea on how to improve this (as I do not have enough quotes and they keep repeating because of the small tag space ), I will try to do my own inverse index method (elastic search is too memory expensive and over-encumbered for my purposes) and then combine it with a big text corpus (free internet e-books). I think I will do something about the porn bots by implementing some kind of spam detection (i have some ideas about this too but implementing them it’s not that exciting for me)

This is an overview of what I did (there are a lot more stories about the process, but I will write them in other future articles). Feel free to drop me questions, or chat about AI stuff.