Jibble

Author Topic: Neural networks and computer art (Inceptionism, DeepDream)  (Read 4062 times)

Snarky

  • Global Moderator
  • Posts: 6,750
  • Private Insultant
    • I can help with proof reading
    • I can help with translating
I brought this up in the AI Singularity thread, but maybe it's something that deserves its own topic.

Researchers recently figured out a way to use "deep learning" neural networks (as used in facial and other image recognition) to instead generate pictures: http://googleresearch.blogspot.ch/2015/06/inceptionism-going-deeper-into-neural.html

People, in particular a group called "Ostagram", are already applying this technique, which Google dubbed "inceptionism", to achieve some startling artistic effects.

You can apply the art style of one image to another:
+ =











Or blend them to create more surreal (nightmarish) results:







Using slightly different techniques, you can even have a robot/printer paint a hardcopy of the result:
[embed=560,315]http://youtu.be/IuygOYZ1Ngo[/embed]

I expect that this technology will be generally accessible pretty soon. If this becomes as easy as a Photoshop filter, how do you think it would be used, e.g. for 2D game art? What if you could take a photo or a simple background sketch, and run a "Bill Tiller" filter on it to make it look like Monkey Island 3?
« Last Edit: 13 Apr 2016, 12:57 by Snarky »

CaptainD

  • Posts: 4,743
    • I can help with play testing
    • I can help with story design
    • I can help with voice acting
    • CaptainD worked on a game that won an AGS Award!
    •  
    • CaptainD worked on a game that was nominated for an AGS Award!
Wow some impressive and interesting results there.  Before I got to your paragraph after the pics I was thinking about how easy this could make creating the graphics for an AG.  That sausage man gave me the creeps though!

Mandle

  • Posts: 7,891
  • NO PIXEL LEFT BEHIND!!!
    • Mandle worked on a game that was nominated for an AGS Award!
Those are some mind-blowing images!

I imagine that in the infancy of this technology to achieve such perfect images takes a lot of know-how, tweaking, and trial-and-error...

But...Wait a bit for development and god only knows what kind of art we might make!

Thanks for the thread Snarky, and for my blown mind!

moloko

  • Posts: 101
  • I wrote a personal text once. It was awful.
    • I can help with AGS tutoring
    • I can help with animation
    • I can help with backgrounds
    • I can help with characters
    • I can help with making music
    • I can help with proof reading
    • I can help with story design
    • I can help with translating
    • I can help with voice acting
    • I can help with web design
I've been following these guys for a while. If you fancy an interesting read, try this.

Snarky

  • Global Moderator
  • Posts: 6,750
  • Private Insultant
    • I can help with proof reading
    • I can help with translating
That's the article from the AI singularity thread I referenced in the first post; please let's keep discussion about that over there.

Danvzare

  • Posts: 2,750
  • The Man with No Name
    • I can help with AGS tutoring
    • I can help with proof reading
    • I can help with scripting
    • I can help with voice acting
Hmm... but can it handle pixel art?
Or you know... pseudo pixel art.

Well even if it can't, this is some pretty interesting stuff. I do wonder how far we'll be able to push it though.

selmiak

  • Posts: 3,877
  • ǝsıɔɹǝxǝ ʞɔǝu puɐ uıɐɹq
    • I can help with play testing
    • I can help with proof reading
    • I can help with translating
    • I can help with web design
    • selmiak worked on a game that was nominated for an AGS Award!
I've seen this recently on another blog. here.
The first render result is very fascinating:

After scrolling through I was mindblown but also got to think this looks like and will end up as a very advanced photoshop filter.
You still need the art and the style as the input, so the computer can catch some patterns and apply these but is not very creative on its own. Some reasults are pretty cool, but still this fucking devalues the artist. the human spirit imagines these artworks and creates it by hand and the machine just eats it.

Ark

  • Posts: 41
Quote
Some reasults are pretty cool, but still this fucking devalues the artist. the human spirit imagines these artworks and creates it by hand and the machine just eats it.
I agree.
Advanced technologies like that make the work of a physical artist cheaper. So as the result artists don't have a choice other than using all these new tricks to save the time and earn more money. Seems it's an unavoidable process.

But I must tell I would use this trick all the time :-D . It's especially useful for monotonous stuff like 100-screen long maze or a long-long desert.

Danvzare

  • Posts: 2,750
  • The Man with No Name
    • I can help with AGS tutoring
    • I can help with proof reading
    • I can help with scripting
    • I can help with voice acting
Quote
Some reasults are pretty cool, but still this fucking devalues the artist. the human spirit imagines these artworks and creates it by hand and the machine just eats it.
I agree.
Advanced technologies like that make the work of a physical artist cheaper. So as the result artists don't have a choice other than using all these new tricks to save the time and earn more money. Seems it's an unavoidable process.

Those darn machines keep taking our jorbs!
Dey took our jab! (laugh)

Personally, I don't think this type of software will be replacing actual artists any time soon.
After all, you still need two pictures to make the picture you want. Not only that, but it can't make something unless you have a picture of it.
I mean, let's say you want it to make a nice futuristic picture with a flying car in the background. What do you do? Pass in a picture that someone else drew that already contains all of that? Not only does that sound wrong (using someone's work without their permission) but you still require an artist to have drawn that picture in the first place. And since it would have already been drawn, why not just use that?

As far as I can tell. This will merely be a tool to supplement the creation of art.

Retro Wolf

  • Posts: 940
  • Hello this is uhhh... Steve... from Microsoft...
    • Retro Wolf worked on a game that was nominated for an AGS Award!
Maybe one day all you'd have to to is type "rogue-like", "puppies", and "Scifi" into games development software and it will make it all for you.

Mandle

  • Posts: 7,891
  • NO PIXEL LEFT BEHIND!!!
    • Mandle worked on a game that was nominated for an AGS Award!
It seems that what the program does is look for similar textures and forms in both pictures and then apply them from one to the other in progressive layers. In the portrait of the guy's face you can clearly see the folds from the woman's dress and the tablecloth being re-used in several places on his face.

So yeah: it is a very, very sophisticated graphics filter, but it's quite a stretch from that to worrying that "art" is being created by machines.

For example: In the first pictures Snarky posted: When I look at the actual painting of the street scene I get a true feeling of how the artist felt about this view and the particular and individual beauty of the elements that they thought were worth capturing and accentuating on their canvas. But...when I look at the computer generated scene which has borrowed this artist's vision out of context I just see a technically well-painted picture by a person talented in copying a real artist's physical techniques of brushstrokes etc, but completely without any particular elements that drawn my eye here and there and make me understand why the artist wanted to paint this picture.

Artistically, it's just a confusing mess.

Like any photoshop filter it is a tool that a human artist can choose to apply: The "art" bit takes place in the way the tool is used, not by its very use.

Anyone can use the "charcoal sketch" filter to change a photo into what looks like something drawn by hand, but you won't see it hanging in a gallery next to pictures that actually were drawn by hand with charcoal, because "art" is not about how good the final product looks, it's about how a human being applied their talents to create something from a vision that started in their head, and recreated that image as a tangible object in physical reality.

This is always an example I used when people fawn over paintings that "look just like photos", or also when they comment that "we have photos, so why paint a picture of someone's face anymore?". A photo is 100% clinical representation what that person looked like at that particular moment...But a painting is what that person looked like to the person painting the picture at that point in time, so it contains the reality of the person's appearance, but influenced by the way the artist personally feels about the model...The image goes through the brain of the artist and is interpreted in the "fuzzy" and emotional way our brains work, before being brought back into the real world via the skill of the artist's hands, also controlled by the same brain. This is why photos will never make portraits obselete.

I'm rambling a bit and saying stuff that everyone here probably already understands, but it's just so frustrating sometimes to hear:

"That's the best painting I ever saw. It looks exactly like a photo of the real thing!"
« Last Edit: 14 Apr 2016, 13:37 by Mandle »

Snarky

  • Global Moderator
  • Posts: 6,750
  • Private Insultant
    • I can help with proof reading
    • I can help with translating
Some reasults are pretty cool, but still this fucking devalues the artist. the human spirit imagines these artworks and creates it by hand and the machine just eats it.

I imagine they said the same thing when photography was invented.

Like others are saying, this is just another tool, like photographs or 3D modeling or Photoshop. Both fine artists and commercial illustrators have always taken advantages of tricks and shortcuts to make things easier for themselves (including things like standard templates that could be individually customized, assistants drawing in the easy bits, camera obscura and mirror tricks, and more recently photo reference and collage/xerography).

Perhaps with this some art jobs will go away, like some work in layout/design and typesetting went away when people could easily do it themselves digitally. But it's not like the world is lacking for designers or typographers! Artists will adapt.

It seems that what the program does is look for similar textures and forms in both pictures and then apply them from one to the other in progressive layers. In the portrait of the guy's face you can clearly see the folds from the woman's dress and the tablecloth being re-used in several places on his face.

So yeah: it is a very, very sophisticated graphics filter, but it's quite a stretch from that to worrying that "art" is being created by machines.

For example: In the first pictures Snarky posted: When I look at the actual painting of the street scene I get a true feeling of how the artist felt about this view and the particular and individual beauty of the elements that they thought were worth capturing and accentuating on their canvas. But...when I look at the computer generated scene which has borrowed this artist's vision out of context I just see a technically well-painted picture by a person talented in copying a real artist's physical techniques of brushstrokes etc, but completely without any particular elements that drawn my eye here and there and make me understand why the artist wanted to paint this picture.

Artistically, it's just a confusing mess.

Well, but partly that's because this is still early days, and partly it's just the deficiency of a not-very-interesting photo combined with a not-quite-suitable technique.

It doesn't seem at all hard to imagine that neural networks such as this could learn to identify and match features in the style and subject images more intelligently, so that you don't get a face made out of tablecloth (although that's an interesting effect in itself), but painted in the same style as specifically the faces in the source image. Something like what they did half-manually in the Rembrandt example. It could also quite conceivably measure depicted proportions and other features of the art, and compare to a database of photos to identify characteristic distortions (e.g. to understand how van Gogh depicted skin tones, or how Picasso abstracted, distorted and rearranged anatomy). And you could imagine an interactive version, where you could mark bits that don't work and have the computer come up with another attempt.

Then by choosing/composing a good subject image, and by selecting a good source style image (or set of images, perhaps), and by doing a bit of cleanup, I don't see why you couldn't create something that looks perfectly convincing, and potentially just as good as the source images. (Depending on how artistically brilliant they were in the first place, and your own artistic talent.)

There will still be human involvement, but it'll be a lot less work. And I think it does make a difference if someone could, from a good portrait photo and a database of paintings, with maybe half an hour's work, create a work that – if not as brilliant as a Vermeer or a Rembrandt – is almost indistinguishable from and the equal to a good classic portrait.

Of course, when anything is possible with the click of a button, deciding what to do is where the talent and skill comes in. Much like a camera can at the click of a button create a picture of anything in the real world, but it's still not at all trivial to shoot a good picture. Or like how movies nowadays can digitally tweak pretty much every aspect of a shot, but some still look much better than others.

Anyone can use the "charcoal sketch" filter to change a photo into what looks like something drawn by hand, but you won't see it hanging in a gallery next to pictures that actually were drawn by hand with charcoal, because "art" is not about how good the final product looks, it's about how a human being applied their talents to create something from a vision that started in their head, and recreated that image as a tangible object in physical reality.

I'm not sure that's true. The reason you don't hang those Photoshop-filtered images in galleries is that they're not as good (and to a certain extent that it's not in fashion). It is the final product that matters. In principle there's no reason why you couldn't hang a work like that in a gallery (and in fact there already are many gallery artworks that have been created digitally, including using semi-automatic and randomized processes). The process is pretty irrelevant, apart from how it shapes the final work.

This is why photos will never make portraits obselete.

To a great extent they already have. Portrait painting used to be a huge industry. But nowadays how many people get their portrait painted, not counting sidewalk caricatures? There is still a market for it, but it's orders of magnitude smaller than it was.

Mandle

  • Posts: 7,891
  • NO PIXEL LEFT BEHIND!!!
    • Mandle worked on a game that was nominated for an AGS Award!
So yeah: it is a very, very sophisticated graphics filter, but it's quite a stretch from that to worrying that "art" is being created by machines.

For example: In the first pictures Snarky posted: When I look at the actual painting of the street scene I get a true feeling of how the artist felt about this view and the particular and individual beauty of the elements that they thought were worth capturing and accentuating on their canvas. But...when I look at the computer generated scene which has borrowed this artist's vision out of context I just see a technically well-painted picture by a person talented in copying a real artist's physical techniques of brushstrokes etc, but completely without any particular elements that drawn my eye here and there and make me understand why the artist wanted to paint this picture.

Artistically, it's just a confusing mess.

Well, but partly that's because this is still early days, and partly it's just the deficiency of a not-very-interesting photo combined with a not-quite-suitable technique.

It doesn't seem at all hard to imagine that neural networks such as this could learn to identify and match features in the style and subject images more intelligently, so that you don't get a face made out of tablecloth (although that's an interesting effect in itself), but painted in the same style as specifically the faces in the source image.

Yeah, but that gets back to the main point that I was trying to put forward through all my rambling:

When you look at a good painting you are looking at a representation of how the artist saw the scene. So...they found it beautiful, or striking, or disturbing, or whatever: It sparked an emotional response which they wanted to share with the viewer...

This is the reason why they used particular colour tones, or brushstroke styles, or went at it meticulously or passionately...And so the particular style of a single great painting contains all the context of this process...

Then, when the viewer looks at the painting their eye is drawn to the features that the artist felt they wanted to really hit you with: The whole reason why they wanted to paint the picture in the first place.

Just painting a different image in the same style is never going to be art, because it lacks the context of the original picture, whether it is done by an amatuer artist just practicing new techniques, or by a computer program, no matter how precise it gets at matching the new subject matter to the desired technique...

There is no "why" in these scenerios, and so there is no art, only replication for no reason...

I did read your whole post and understand that you also addressed these issues further, so I think we kinda agree mostly on the issues...

I just thought that this one section of your post was something I could respond to to maybe clarify what I was trying to say earlier.

Snarky

  • Global Moderator
  • Posts: 6,750
  • Private Insultant
    • I can help with proof reading
    • I can help with translating
Well, that comes down to what you define as "art" (you seem only to consider fine art), and whether art has to great art to be art at all. I wouldn't normally think so.

I tend to think anything can be art, and that it's more a question of the attitude of the audience than the processes that created it. The idea that a lot of human personality and deliberation has to go into a work for it to be meaningful to audiences (that you need an active artist) is at best an assumption, and one that I think technologies such as this will in fact increasingly challenge.

There is always at the least a selection and curation process that defines what we approach as art. So I think it's valid to say: "I had the computer generate 2000 different image mashups, and this one came out to something really interesting" and have that be a piece of art.

miguel

  • Posts: 2,372
  • Onions? Give me back my onions!
    • miguel worked on a game that was nominated for an AGS Award!
But nothing beats the emotional courage of facing a blank canvas, there's a responsibility to the artist when painting somebody's face that it simply vanishes when clicking a button.
A true artist steps to the ring and assumes his work, he puts emotions on a canvas until he signs and after it's finished there's no software to blame for.
I am with Mandle on this, although the technology shown is pretty cool.

Enviado do meu D2303 através de Tapatalk

Working on a RON game!!!!!

Danvzare

  • Posts: 2,750
  • The Man with No Name
    • I can help with AGS tutoring
    • I can help with proof reading
    • I can help with scripting
    • I can help with voice acting
A true artist steps to the ring and assumes his work, he puts emotions on a canvas
So in short, all we need to do to make you consider this art, is make computers feel emotion.
Well that should be easy enough. The only question that remains is how many years are left until someone does that. Definitely under a hundred if you ask me.

miguel

  • Posts: 2,372
  • Onions? Give me back my onions!
    • miguel worked on a game that was nominated for an AGS Award!
I'm not that radical, art can and will come out using that technology I don't doubt. It's just not art as I perceive and that's obviously self-sided.
I also don't agree with computers feeling anything really, humans feel, robots don't and it should stay that way. It's great to read a sci-fi novel about robots with human traits but having them around is just weird for me. I love Blade Runner but those replicants are some scary dudes and dangerous. We have enough crazy people exploding themselves without the help of robots.

Art and humans are connected and some people even consider art to be what makes us humans. Technology changes and that's great but true art is only understood by humans like you and me even with our different tastes. No robot will ever "understand" art like humans do, it can emulate what we do and think but that's completely different.
We can feed a computer with all the great phrases ever written by literature masters but that will not make a good book. The same with music, painting and all of artistic expressions there are.
Working on a RON game!!!!!

Scavenger

  • Cavefish
  • Posts: 956
  • In Four Glorious Colours!
    • I can help with animation
    • I can help with backgrounds
    • I can help with characters
    • I can help with scripting
    • Scavenger worked on a game that won an AGS Award!
    •  
    • Scavenger worked on a game that was nominated for an AGS Award!
Art and humans are connected and some people even consider art to be what makes us humans. Technology changes and that's great but true art is only understood by humans like you and me even with our different tastes. No robot will ever "understand" art like humans do, it can emulate what we do and think but that's completely different.
We can feed a computer with all the great phrases ever written by literature masters but that will not make a good book. The same with music, painting and all of artistic expressions there are.

Well, that's just it - art is only art if it is percieved by a human. A computer doesn't have to appreciate art, it only has to create it for humans to appreciate. And humans can imprint meaning on the meaningless just as well as they can see meaning in something intended.

Art is art if someone looks at it and has an emotional reaction to it. They don't need to see an artist's singular brush strokes doing whatever the heck thing you'd ascribe to a brush stroke. Art is bigger than that. A human could press a button a thousand times and then see something beautiful and then say "that's art" and it will be.

Also, literature made by a computer isn't as coherent as human written literature, but it sure is artful in it's own way:

Code: Adventure Game Studio
  1. VIOLA:
  2. Why, Salisbury must find his flesh and thought
  3. That which I am not aps, not a man and in fire,
  4. To show the reining of the raven and the wars
  5. To grace my hand reproach within, and not a fair are hand,
  6. That Caesar and my goodly father's world;
  7. When I was heaven of presence and our fleets,
  8. We spare with hours, but cut thy council I am great,
  9. Murdered and by thy master's ready there
  10. My power to give thee but so much as hell:
  11. Some service in the noble bondman here,
  12. Would show him to her wine.

Now that's art.

miguel

  • Posts: 2,372
  • Onions? Give me back my onions!
    • miguel worked on a game that was nominated for an AGS Award!
I really don't know what art is, I Base my knowledge of it by studying art through books and video documentary, I also paint in oil and pastels and occasionally sell my paintings.

As I understand the world, humans create artistic expressions of their experiences, and a computer program designed to make a illustration of somebody's experience is as valid as any other means.
Please don't take me as the old conservative man that I sound like, art is indeed free to be understood as people want and like.
My point is that there's no possibility of a computer to create a unique masterpiece even remotely.





Enviado do meu D2303 através de Tapatalk

Working on a RON game!!!!!

Danvzare

  • Posts: 2,750
  • The Man with No Name
    • I can help with AGS tutoring
    • I can help with proof reading
    • I can help with scripting
    • I can help with voice acting
I really don't know what art is, I Base my knowledge of it by studying art through books and video documentary
Yeah, I doubt any of us here truly know what "art" is.
As far as I can tell though, there's several definitions. The two most typical ones are the "everything is art" and the typical "only certain things are art". Both are equally acceptable. If anything, I believe most of us live by both definitions. We understand that everything is art, no matter what it is, but to us only certain things are "art". If you get what I mean.

My point is that there's no possibility of a computer to create a unique masterpiece even remotely.
It is my opinion that if given enough time, a computer will be able to do anything a human can do, and probably even more.
A good example is that a computer currently can't feel emotions (and in my opinion, shouldn't), but it doesn't mean that we can't make a computer feel emotions. Heck, it should even be possible to make a computer feel emotions that humans can't!

Also, here's something else for you to think about and to add further fuel to the fire.
There is absolutely no possibility of me making a unique masterpiece, not even remotely. Does that make me a computer?