I do colorizations on Reddit. How about adapting this to assist manual colorizers? We'd mask out each color region by hand, so the neural net wouldn't have to decide what's what. We'd tell it, "this region is skin, this is a brown overcoat, this is a gold candlestick - please colorize them appropriately." As it is, colorizers fill in those regions with a very small number of colors, sometimes with gradient maps that blend two or three colors together, but there's just not much color variety in even the best works. Grass should be a hundred shades of green, but we'd use 2 or 3. Skin is the hardest to get right, and requires more individual hues, but that means maybe 5 to 10. It's enough to trick the eye, but on close examination looks more painterly than photographic. A neural net palette picker could be Photoshop's next big feature. An intelligent skin colored crayon might actually deliver all the shades of skin.
"I do colorizations on Reddit. How about adapting this to assist manual colorizers? We'd mask out each color region by hand, so the neural net wouldn't have to decide what's what. We'd tell it, "this region is skin, this is a brown overcoat, this is a gold candlestick - please colorize them appropriately."
That's a smart idea. Master painters used to do this. Do the broad outlines, choose the colour and style then unleash the underlings (painters in training) to do the rest.
If you could figure out a fleshed out version of the interaction loop - "this is skin, no, not like that, a little more like this" - that would probably be useful. The interface between AI and people is always a little weird - how could you tell it to do a small correction and have it be meaningful? What do two to three AI driven actions a second look like?
Thing you are talking about is similar to scene labeling (google the term and look at the pictures). Only you'd be doing it by hand, instead of letting NN find it's own representation. This is good because you'd be creating a dataset full of regions that we could use to automatically segment the pictures into useful regions later.
There's even a language mapping that you want, you want it to recognize image parts and associate it with a word, which also isn't simple, because you'd have to have a lot of labels (candles, hands, hairs, cars, trees, licence plates etc.)
It is a harder problem than the one in the article.
State-of-the-art scene labeling is still not good enough (close to 80% accuracy) but I believe it's due to lack of data because algorithms used combine neural networks with joint learning approaches such as conditional random fields to extract the regions.
