Stylistic Rendering Using an Arbitrary Texture Set

A Project By Maxim Garber

Project Outline

Goal

I will be developing an interactive system which when given some 3D geometry, a lighting specification and a finite set of textures, will render the scene, mimicking the shading on the objects in a non-photorealistic style using only textures from the texture set. For example, if the system is given a sphere model, and a texture set of various floral patterns, then it should be able to render a floral patterned sphere with textures placed to duplicate as much as possible the shading of the sphere from the lights.

Approach

I plan to use a two phase approach in which the first phase is a learning step that separates up the texture set and decides a priori which texture will be used for polygons based on criteria such as colour, intensity etc. This dividing criteria may be a simple function or may involve some machine learning techniques like neural nets. The goal is to ensure that a polygon is textured in such a way as to best mimic the colour and intensity of that polygon in the standard shading model. The second stage will involve rendering a given scene by bucketing polygons and rendering them each with their best matching texture. This will involve some computation per vertex and evaluation of which texture best matches the intensity at that vertex, as well as some blending or subdividing of triangles whose vertices match with different textures.

Motivation

Common non-photorealistic systems tend to try and mimic one specific rendering style. This system should provide a method for users to easily generate the non-photorealistic rendering styles that they want by specifying a set of textures.

Personal Interest

I have always had some interest in exploring non-photorealistic rendering, especially in using the computer to create new rendering styles instead of just mimicking existing styles like cartoons or line art. Also, as a change, I want to work on a project that gets away from the purely geometric work that I do in my research.

Novel Component

As far as I know, previous work in NPR has focussed on mimicking specific artistic or cartoon styles. The flexibility of the system will be novel. Also, the possible use of neural nets for NPR, and maybe even graphics in general, has not been that fully explored.

Previous Work

There has been previous work done in using texture mapping to produce various non-photorealistic styles such as cartoon shading and line art.

References:

Lake A., Marshall C., Harris M. Blackstein M., “Stylized Rendering Techniques For Scalable Real-Time 3D Animation” Intel Architecture Lab & University of North Carolina.

Salisbury M., Wong T., Hughes J., Salesin D., “Orientable Textures for Image-Based Pen-and-Ink Illustration”, University of Washington.

Ideas about matching image intensity with another image or texture is also used in work on Dithering and Photomosaics.