Extended depth map features – goal met!

Over at  Sprite Lamp’s Kickstarter campaign, we have just hit our next stretch goal at 20k! This is pretty awesome. This means extended depth map features, so it’s time to give a big rundown of how this will work. I mentioned two aspects of this – they are depth map editing from within Sprite Lamp, and exporting meshes that are based around the depth map.

Depth map editing

Currently, Sprite Lamp is able to generate a depth map from a normal map. This feature is most visible in the examples I’ve posted in the form of self-shadowing, particularly this image of a stone wall, which I’m going to post again because I love it so much:


If you look closely, Sprite Lamp has caught the depth quite accurately, right down to the little notches in the stone, and the bits where there are weird mortar shapes left on. Feel free to inspect the detail:


While it’s important to remember that there’s no such thing as a ‘correct’ depth map based on a drawing, I think the above image is pretty good. In general, I’m pretty pleased with Sprite Lamp’s algorithm/system for generating a depth map from a normal map. This is a pretty difficult problem, made worse by the fact that you get a physically imperfect normal map when it’s drawn by hand. Textures like the one above are, fortunately, a pretty good case for this application. Alas, not everything can be a best case scenario. I won’t go into too much detail here, but the worst case scenarios for Sprite Lamp’s depth generation are all about discontinuities in the depth map. They happen when what you draw has one object passing in front of another – it’s hard for Sprite Lamp to guess how far in front. This doesn’t happen much with textures (like the stone wall, above) but it does happen with character art. Sprite Lamp gets pretty good results here too – good enough for some nice self-shadowing effects, as I think is evidenced by the sample art on the main page and in the Kickstarter.

However! With algorithms like these that involve guessing what the artist intended, there are always times where you guess wrong. For that reason, I’d like to give the user the ability to mess with Sprite Lamp’s ‘interpretation’ manually a little bit. Now, naturally, I’m not going to offload all the dirty work onto the user – if I was going to do that you might as well just draw it by hand. The goal is small and intelligent tweaks.

With that in mind, I have a few features that I’m going to experiment with to make working with depth maps better. I’m not saying these will all make it into the final program – these are what I’ll try, but I don’t yet know what will be worth including and what will end up being a waste of time.

  • Edge detection: Sprite Lamp will go through the lighting profiles and try to figure out where depth discontinuities might be.
  • Silhouette maps: This is a plan I have to have the artist draw a simple map that will be deliberately easy for Sprite Lamp to detect the edges of. May or may not end up being necessary, but will mostly be trivial to create because it comes from mask layers that come from the drawing process anyway.
  • Edge-aware soft selection: The first two points here are ways of figuring out where depth discontinuities are – this is useful information to have, because it allows the user to easily select areas of the depth map without going ‘outside the lines’. If you have a foreground object and a background object, this will allow you to grab the depth values of the foreground object and move it back and forth without interfering with adjacent pixels (which are the depth values of another object entirely).
  • Spring systems representing depth values: This will allow the user to drag a single depth pixel forward and back, and have adjacent pixels come with to a greater or lesser extent. There won’t be springs connecting pixels along discontinuity edges.
  • Curve editor to do an image-wide readjustment of values: This is something similar to how colour readjustment works in programs like Photoshop, and it could indeed be done in those programs – however, I think it might be worth including it in Sprite Lamp because getting immediate feedback on how it looks will make things a lot easier to deal with.

So, that’s that. I’m looking forward to playing around with these things, and of course I’ll keep you posted as to how it all goes. As for the other part of this stretch goal…

Mesh Exporting

The other part of this stretch goal is to add the ability to export meshes. This doesn’t take quite as much explaining as the other feature, though it will take a bit of work on my part. The goal here is to export an intelligently-created mesh that takes the shape of the object according to the depth map. Naturally, the easy approach to this is to simply subdivide a quad into many quads, then displace the vertices according to the depth map. I’m hoping I can do quite a bit better than that, by placing vertices in important places along ridges and points, and making the geometry more sparse in flatter areas. The user will have a slider that allows them to adjust the target vert count of the generated mesh. This has a couple of concrete uses:

  • Stereographic rendering without relying on complex shaders: Naturally, I quite enjoy complex shaders, but there are times when it’s easier to just use geometry. This can save you from various annoying consequences of displacement shaders, such as correctly handling extreme angles.
  • Correct rendering into depth buffers for ‘true’ shadowing: I’m going to be writing a lot about the various sneaky tricks you can use to fake shadows using depth maps – however, there are certain shadowing algorithms where it’s just nicer to do things with geometry. Having geometry at your disposal gives you more options for plugging directly into existing ‘general’ shadowing options, such as shadow mapping and CSMs, which are well-documented already and amply implemented in existing engines.
  • As a base for further tessellation/displacement: The user will be able to determine how strong the vertex displacement will be, and that will include zero – that means that you can use the flat mesh as a basis for more effective GPU-based tessellation in the GPU, and displace generated verts based on the depth map in a shader.

In addition to these, though, I’m looking forward to see what else people come up with using this feature. I’m certainly planning on making some 3D prints of artwork once I get it done!

One last thing I’ll say on this subject is that I haven’t settled on formats for exporting of meshes. I’ll probably make use of the dubiously-named AssImp (short for ‘Asset Importer’, obviously), which can export to a handful of formats, in accordance with this list. However, if anyone knows of any straightforward mesh exporting libraries for C# you think would be better, I’m all ears.

Stretch goals and PayPal added to Sprite Lamp campaign

So I sent out a survey to the early Sprite Lamp backers, asking about what kinds of features they’d find useful, and got a pretty extensive and pretty quick response. So without further ado, I present to you Sprite Lamp’s stretch goals.

StretchGoal_6kThere are more details available on the Kickstarter page itself – scroll all the way to the bottom (almost). These stretch goals are bigger than the original goal by such a margin because they represent new features that I have to build from the ground up, now – the original goal represented remaining costs of a significantly pre-developed project.

Other things I learnt are that Sprite Lamp backers are pretty tech-savvy, with 24% saying they’re completely confident they can make their own shaders, and only 5% saying they have no chance of doing so. 24% are rolling their own game engine, and an impressive 70% knew what I was talking about in the question about anisotropy maps. I also learnt that almost half of the backers were planning on using Unity in conjunction with Sprite Lamp, which isn’t all that surprising, that TextureMapper is an overwhelmingly popular atlas tool, and that Spine appears to be the 2D skelmesh software of choice. Finally, I learnt that pretty much everyone is at least intrigued in trying out the palettisation tech that I have in mind.

Thanks to everyone who helped me out by responding to the survey!

Sprite Lamp hits its goal!

Well, I was planning on posting an update when Colour Bind’s Kickstarter went live, but I ended up getting distracted by a bunch of social media stuff, and now that I am actually getting around to posting this, it has a different title.

Sprite Lamp on Kickstarter

That wasn’t quite what I was expecting. Sprite Lamp hit its goal of 6k around the seven hour mark, which I strongly suspect (but can’t prove) makes it the first official Australian kickstarter to be funded. Sweet! The obvious thing to say is THANK YOU to everyone who pledged, and everyone who helped boost Sprite Lamp by sharing the kickstarter on social media (and in general). Because of you, Sprite Lamp will not only happen, but it will happen in a timely fashion, and most likely it will get a bunch more features and be a higher quality product than it would have otherwise been. Also because of you, I get to buy a new pair of shoes (seriously, my old shoes have holes in them).

With that out of the way, it seems I have some work to do. I had put some vague stretch goals in the Kickstarter, but now I’ve got to make them specific much more quickly than I was expecting. I’ve already had some good suggestions. In accordance with one such suggestion, I’m going to set up some rudimentary forum here for talking about suggested features and the like.

I’m also going to make a series of blog posts here about a variety of graphics techniques that will be applicable to Sprite Lamp. I’m going to cover things like how the shaders work, issues with using normal maps that you might not see coming (like the fact that rotating a normal map in Photoshop results in incorrect normals), and a variety of shadow techniques that exist somewhere along the correct/fake spectrum that might work well in conjunction with Sprite Lamp.

Oh, and don’t forget that I need your precious votes on Steam Greenlight – any help there is much appreciated.

Thanks for all the support, everyone!