Petroglyph digitization: enabling cultural heritage scholarship

George V. Landon and W. Brent Seales, “Petroglyph digitization: enabling cultural heritage scholarship,” Machine Vision and Applications, vol. 17, no. 6, pp. 361–371, December 2006.

Abstract:
The digitization of antiquities is facilitating a renaissance for scholars who have unprecedented access to rich representations of objects. Cultural Heritage digitization is a central challenge, and its subtleties are intertwined with object properties and the constraints of physical access and handling. In this paper, we present the design and analysis of a system built for the digitization of Puerto Rican petroglyphic iconography. The petroglyphs exhibit unique properties (shape, size, surface) that determine system design choices. The 3D models obtained with the system support new scholarly and educational activities, including interactive surface lighting, feature highlighting and annotation through mark-up, and immersive viewing using large-scale displays.

Available Online

Computer Science & Illusion

As noted on Robert Burke's blog, Trinity College Dublin (TCD) recently hosted a great lecture by Prof. Andrew Blake. He really runs the gambit on topics in computer vision covering various theories and applications.

Part 1:


Part 2:

If you prefer not to stream: Full Movie Download (118MB WMV).
Also, the PowerPoint slides are available (44MB ZIPPED PPT).

Is Computer Science losing its coolness?

The Seattle Times has just run an interesting story, titled Where'd The Whiz Kids Go?, on the lack of CS students in Washington Universities. It seems that this is a nation-wide problem that has gotten big enough to get the attention of politicians.

From the article:

"... despite the seemingly limitless potential of computers, educators are having a tougher time than ever convincing students to pursue the field. It can be hard work. Boring, even. And there's that enduring, if unfair, image problem. Picture the socially inept geek hunched over a screen at 3 a.m., Coke in hand, pecking away at pages of incomprehensible code.

'There was such a boom of interest in the '90s, and now you get the sense around the country that computer science is past its prime. But the most exciting stuff is still in front of us.'

Meanwhile, Microsoft continues to add workers locally at the rate of 4,000 a year. In this year's record class of 5,400 UW freshmen, 300 say they're hoping to graduate in computer science or engineering. Even if none dropped out or changed majors, the class of 2010 wouldn't amount to a month's supply of new workers needed just at Microsoft's Redmond campus."

There certainly does not seem to be an easy solution to this shortage. It is also interesting to note that the ACT 2006 National Report shows that 2.05% of test takers are interested in at least a 4-year Computer Science or Math degree.

Interactive Tabletop Museum Exhibits

In the most recent issue of IEEE Computer Graphics and Applications, there is an interesting article titled "Interactive Tabletop Exhibits in Museums and Galleries."

From the article:

"The museum experience is an unusually tactile, sensual one, and the standard keyboard-mouse-and-screen setup might seem out of place. This trend toward sensual involvement is particularly noticeable in tabletop displays, as they appeal to two aspects of familiar daily life: the horizontal surface as a workspace, and hand gestures (or common objects) as tools for manipulating information."

The full article can be viewed online (PDF). Readers should notice that the article contains a nice sampling of various interactive/tabletop exhibits. The Tilty Table, in particular, demonstrates that interactivity and intuitive access can (and should) be combined in the museum experience. A sample video is available (MOV).

Smart Video Surveillance

The Journal News has conducted an interview with Arun Hampapur manager of IBM's Exploratory Computer Vision Group. The focus is mainly on how the 9/11/2001 attacks changed their research focus in computer vision.

From the Interview:

"Q:How did 9/11 affect your field?
A:Pre-9/11, the success stories in computer vision were around machine vision: How you inspect a printed circuit board while it is being manufactured. Now the biggest application of computer vision would be security. And there are two pieces to that security puzzle. Biometrics is answering the question, 'Who is this person?' Surveillance is answering the question, 'What is going on?'

Q:How does it work?
A:You can apply two kinds of functionality. One I called real-time alerts. You have a port and you have a fence, you don't want anyone jumping the fence. Or you have a retail store with a loading dock, and you don't want anyone on the loading dock past 9 o'clock. These are known conditions, for which you can say, 'OK, so if someone shows up on the loading dock after 9, tell me.'The second is being able to find things. Security is a kind of a cat-mouse game. Sometimes something becomes relevant only after the fact. If you remember the Washington sniper incident, somebody said there was a white van at the first scene, and then the police spent a lot of energy trying to look for the white van. There was no technology at that time which could use cameras to find white vans."

An overview (PDF) of this work has been published in IEEE Transactions on Signal Processing.

3D Photo Tours

With all the recent web coverage of Microsoft Photosynth, I thought that I should post some of its foundational work here.

From the paper:

"Our system consists of an image-based modeling front end, which automatically computes the viewpoint of each photograph as well as a sparse 3D model of the scene and image to model correspondences. Our photo navigation tool uses image-based rendering techniques to smoothly transition between photographs, while also enabling full 3D navigation and exploration of the set of images and world geometry, along with auxiliary information such as overhead maps. We demonstrate our system on several large personal photo collections as well as images gathered from photo sharing Web sites on the Internet."

From a computer vision standpoint, they claim to be using a modified SIFT for wide-baseline matching. While this does not appear to be revolutionary results, it should be understood that, in this case, presentation is everything. This work appears to have a very intuitive interface that could revolutionize image search/retrieval/browsing. Check out this Interactive Demo of a trimmed-down version. Also, here is a nice video (120MB MOV) from SIGGRAPH.

Full Paper (PDF)

Organic Pixels from Butterflies

While this falls outside the normal areas covered here, it does appear to be some rather fascinating work. Besides, what good is work in graphics without a good display.

From the article:

"The wings of the male Cyanophrys remus are bright metallic blue on one side, thought to attract mates, and a dull green on the other to act as camouflage. ...each side of the wing contained different photonic structures. The metallic blue colour is produced by scales that are photonic single crystals whereas the dull green is the result of a random arrangement of photonic crystals. This randomly arranged structure may have powerful applications. The crystals can actually produce different colours – green, yellow and blue – depending on their orientation, but the overall effect in Cyanophrys remus is a dull green. The team also found a way to make the crystals generate red reflections. The red-green-blue palette could be used for flat-panel visual displays... by making an array of crystals mounted on microelectromechanical arms that could change their orientation. In that way it would be possible for each 'pixel' to produce red, green or blue."

From an outsiders perspective, this is quite amazing. I will leave those more versed in the area to make observations about how novel/unique this work is. The results, published in Physical Review E, can be found in this paper (PDF).

Google's Manual Image Recognition

Google has recently added a collaborative Image Labeler application to their image search.

From the site:

"Google Image Labeler is a new feature of Google Image Search that allows you to label random images and help improve the quality of Google's image search results. Each user who wants to participate will be paired randomly with a partner who's currently online... Over a 90-second period, both participants will be shown the same set of images and asked to label each image... based in part on technology licensed from and developed at Carnegie Mellon University."

This is actually a fun little game to play. However, it seems to be missing some depth. The work this is based on, by Luis von Ahn at CMU, is a game first and an information extractor second. He has written about this in Computer (PDF) and has also published (PDF) the work. While this certainly seems like a good use of abundant Web users, it is an interesting turn considering their recent acquisition of Neven Vision as discussed on Google's blog. I wonder if at some point Google will implement user verification of automated metadata extraction or if these two paths will stay separate.

Flatland: A Romance of Many Dimensions

Google has started putting some really great books online. They listed a few noteworthy titles in a recent blog post.

From the Official Google Blog:

"...you can go to Google Book Search and download full copies of out-of-copyright books to read at your own pace. You're free to choose from a diverse collection of public domain titles -- from well-known classics to obscure gems."

One title, from the blog post, should really stick out to Computer Vision and Computer Graphics researchers; Edwin Abbott's Flatland: A Romance of Many Dimensions. While the mathematical concepts in the book are fairly elementary, it is a great change of pace from typical textbooks. If you have or are planning on doing any work that involves projective geometry this is a must read. Also, the social satire included by Abbott adds nice context to his work.

As an aside, I recently found a new movie being produced as an educational unit for young math students. A trailer of the aptly named Flatland: The Movie is available on their website.

Binary-coded Shutter for Deblurring Still Images

Researchers at Mitsubishi Electric Research Laboratories (MERL) have developed a new filter system for digital cameras. The Coded Exposure Photography: Motion Deblurring using Fluttered Shutter uses a shuttering technique that effectively applies a binary-encoded timing filter over a continuous sequence of images.

Full Paper (PDF)

Very nice results. It will be nice to see this, or something similar, introduced in consumer camera.

Insect Techniques for High Dynamic Range Video

Researchers at Adelaide University, Australia are developing Image Processing techniques that replicate the vision system of flies.

From the article:

"...conducting experiments that involved recording the activity of fly brain cells as they were shown different images. 'We were amazed at the extra detail they were able to extract from the dark parts of a scene. This led us to test exactly how they were able to do this and to reproduce the processing in electrical circuits and computer simulations.' Unlike a camera, flies and other animals can tune their eyes to the light levels of different regions of an image independently. 'In nature, the individual cells of the eye adjust to a part of the image independently in order to capture the maximum amount of information about the scene..."
Example Video (MP4)

While the results from the video look promising, I was unable to find any papers on this work, so I am a little hesitant to claim this work as novel. It seems to fall easily under the umbrella of High Dynamic Range Imaging (HDRI). Paul Debevec has done a large amount of work in this area already. If you are interested, He has posted a nice collection of papers and links here.

Mood Dependent Art

It is always nice to see novel combinations of research areas that produce innovative applications. Recently, researchers from the Universities of Boston, USA and Bath, UK have developed such a system called "Empathic Painting: Interactive stylization using observed emotional state." From a Computer Vision perspective, the work includes facial expression recognition. Also, Image Processing and graphics are used to create a real-time painting filter. While these areas are not ground-breaking by themselves, together they make a very unique and creative use of technology.

Links from the site:
Full Paper (PDF)
Example Video (AVI)

This work was published in Proceedings of the 4th international symposium on Non-photorealistic animation and rendering
2006. These proceedings include some other very interesting projects.

Stereo Reconstruction for Improved Automotive Safety

In a recent news release, Toyota describes a new safety system that implements numerous computer vision technologies.

These techniques include(from the article):

"...a newly developed stereo camera to detect pedestrians and support emergency collision evasion maneuvers by the driver. ...three-dimensional object detection information from the stereo camera to detect not only vehicles and obstacles, but also pedestrians. A near-infrared projector located in the headlights supports nighttime detection... The system retracts the seatbelts and warns the driver when it determines a high possibility of a collision. If the driver does not brake, the Pre-crash Brakes are applied to reduce collision speed."

This leads to the question of how long until we have black box recorders in every automobile. Continuous data acquisition from this many sensors would certainly help with accident reconstruction.

Capturing 3D Fluid Surfaces

Researchers at the University of Delaware are:

"...proposing a novel approach for accurately reconstructing three-dimensional fluid surfaces through the design of an experimental system using a light field camera array that can simultaneously capture different views of a fluid surface.
      The light field camera array features a number of digital cameras, from 16 to 128, with specially modified flashes, lenses and apertures. Instead of one flash, each camera is equipped with four.
      ...the system works by placing a known pattern beneath the surface, with each camera in the array observing a distinct time-varying distortion pattern. A sampled fluid surface can then be measured by analyzing the distortions. For surface reconstruction, the researchers plan to develop an algorithm to minimize the error relative to the sampled data."

This should be an interesting project to follow. The previous work that I have seen with multi-flash cameras captured static scenes, so I am curious how Jingyi Yu will modify the system to handle dynamic fluid.

Learning Epipolar Geometry

Sylvain Bougnoux (looks like he's at IMRA-europe now) has written a nice page, featuring interactive Java applets, to assist in learning the basics of Epipolar Geometry. This should be helpful for students just starting out in computer vision. Especially if you are a visual learner and enjoy gaining geometric intuition on a subject.

How to Write a Great Research Paper

Here are some nice thoughts on direction and steps for writing a good research paper. In particular, certain traps that most researchers can fall into. I know for me remembering to write a paper (at least a draft) right after I come up with an idea will give me much better direction when actually implementing/testing the idea.

How to write a great research paper(PDF). (Simon Peyton Jones, Microsoft Research)

Note: Here is the listed reference.

Computer Vision Job Listings

Here are a few site that list Computer Vision related positions to be filled. These are certainly not new, but are listed here for reference. Feel free to add others in comments.

• ECVision: Cognitive Computer Vision Jobs
• Vision Science Jobs
• The VISLIST
• phds.org
  

Keeping an Annotated Bibliography

As a continuation of helpful things for researchers, here is another hint Dr. Marian Petre reminds us to keep in mind when researching/writing.

One of the things that established researchers have is a working knowledge of the relevant literature. Most established researchers have a core repertoire of some 100-150 works on which they can draw readily. These are a useful selection from the hundreds or thousands of articles and books the researcher has digested over time. The annotated bibliography is an effective mechanism for facilitating this acquisition and for keeping record of the majority of papers that fall outside the core.

What the annotated bibliography should include

It should include, as a minimum:

• the usual bibliographic information (i.e., everything you might need to cite the work and find it again)
• the date when you read the work
• notes on what you found interesting / seminal / infuriating / etc. about it. (The notes should not just be a copy of the abstract; they should reflect your own critical thinking about your reading. They can be informal, ungrammatical, even inflammatory as long as they retain meaning about your reading. If you read a paper more than once and get different things from it, then add to the notes but do keep the original notes, which can prove useful even if you've changed perspective or opinion.)

It can include many other useful things, e.g.:

• where the physical copy of the work is (e.g., photocopied paper, book borrowed from the library, book in one's own collection)
• keywords, possibly different categories of keyword
• further references to follow up
• how you found the work (e.g., who recommended it, who cited it)
• pointers to other work to which it relates
• the author's abstract

Generic Viva Questions

I found some useful information for PhD students and researchers in general from the SIGCSE Doctoral Consortium. These are excerpts from a compilation by Dr. Marian Petre, March 1999.

• How did you come to research this topic in this manner?
• What are the main achievements of your research?
• What has your thesis has contributed to our knowledge in this field?
• What are the major theoretical strands in this area: what are the crucial ideas, and who are the main contributors?
• What are the main issues (matters of debate or dispute) in this area?
• Where is your thesis 'placed' in terms of the existing theory and debate? How would the major researchers react to your ideas?
• Whom do you think will be most interested in this work?
• Why did you choose the particular research methodology that you used?
• What were the crucial research decisions that you made?
• If you were doing this research again, would you consider using any other research methodology?
• What do you see as the next steps in this research?
• What was the most interesting finding in your results?
• Isn't this all obvious?
• Were you surprised by any of your results (if so: why, and what was surprising)?
• What advice would you give a new student entering this area?
• What is your plan for publication?
• What haven't I asked you that I should have, and what would your answer have been
  

New Methods for Rock Art Recording and Virtual Analysis

George V. Landon and W. Brent Seales. New Methods for Rock Art Recording and Virtual Analysis. Michael A. Cinquino and Michele Hayward, organizers. Symposium on Latin American Rock Art: Conservation and Research Status. Proceedings of the 71st Annual Meeting of the Society for American Archaeology. 2006 April 26-30; San Juan, Puerto Rico, USA.



Abstract:

We present a method for preserving and visualizing petroglyphic art. The process creates accurate 3D textured models of carved surfaces, making the intricate geometry available for study with interactive computer tools. Examples of useful virtual analysis include visualization with synthetic lighting, simulation to highlight the subtleties of the carvings, and texture-mapping with annotated content. We demonstrate with results from Taino petroglyphs, which are located at the ceremonial center in Caquana, Puerto Rico.

Building and Visualizing 3D Textured Models for Caribbean Petroglyphs

IACA 2005 - 21st Congress of The International Association of Caribbean Archaeology (Trinidad & Tobago) Building and Visualizing 3D Textured Models for Caribbean Petroglyphs

Geometric and Photometric Restoration of Distorted Documents

Mingxuan Sun, Ruigang Yang, Yun Lin, George V. Landon, W. Brent Seales, Michael S. Brown, "Geometric and Photometric Restoration of Distorted Documents," in Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 2, October 2005, pp. 1117– 1123.

Abstract:
We present a system to restore the 2D content printed on distorted documents. Our system works by acquiring a 3D scan of the document’s surface together with a high-resolution image. Using the 3D surface information and the 2D image, we can ameliorate unwanted surface distortion and effects from non-uniform illumination. Our system can process arbitrary geometric distortions, not requiring any pre-assumed parametric models for the document’s geometry. The illumination correction uses the 3D shape to distinguish content edges from illumination edges to recover the 2D content’s reflectance image while making no assumptions about light sources and their positions. Results are shown for real objects, demonstrating a complete framework capable of restoring geometric and photometric artifacts on distorted documents.

Building and Visualizing 3D Textured Models for Caribbean Petroglyphs

George V. Landon and W. Brent Seales. Building and Visualizing 3D Textured Models for Caribbean Petroglyphs. In 21st Congress of The International Association of Caribbean Archaeology (Trinidad and Tobago), July 2005.

Abstract:
We present a system for building and visualizing 3D textured models of Caribbean petroglyphs. Through a collaboration with the Instituto de Cultura Puertorrique~na, we show reconstructions of petroglyphs located at the Taino ceremonial center at Caguana, Puerto Rico. Our methods allow digital access to and preservation of objects located in remote areas that remain unprotected from the elements. Our approach creates 3D textured models and allows users to intuitively visualize subtle aspects of petroglyphic art, such as dimension, texturing, and application of synthetic lighting, that are not possible without the digital model. We discuss the performance of these methods in practice and show resulting high-resolution 3D scans of Taino petroglyphs. We believe these techniques can be useful in promoting access, scholarship and in fostering preservation for artifacts that are inaccessible and obscure.

Full text (PDF)

The Museum and the Media Divide

W. Brent Seales and George V. Landon, "The Museum and the Media Divide: Building and Using Digital Collections at the Instituto de Cultura Puertorriquena," in D-Lib Magazine, vol 11, May 2005.

Abstract:
By digitizing physical holdings, museums have an important opportunity to provide patrons unprecedented access to manifold media. Digitization of existing collections, which is the museum's form of the content creation problem, is a crucial and technically challenging step in continuing to narrow the digital divide between the patron's desire for rich multi-media information and what the museum actually provides. We present a case study of the issues involved in executing a digital content creation program in partnership with the Instituto de Cultura Puertorriquena. We focus on the challenges involved in acquiring, organizing and accessing the collections to make them meaningfully available within typical budgetary and technical constraints.

Available Online