Sketch Recognition Lab Presents to Texas A&M Computing Society

On Tuesday, February 24, 2015, Sketch Recognition Lab members Raniero Lara-Garduno, Seth Polsley, Larry Powell, Paul Taele, and David Turner were on hand to present and discuss several of the lab's latest on-going research projects in front of a room-filled audience of students from the Texas A&M Computing Society (TACS), the representative undergraduate student organization for the Department of Computer Science & Engineering. The Sketch Recognition Lab, directed by Dr.Tracy Hammond, was invited by TACS to present to the undergraduate students as part of a series of guest talks and workshops to improve and broaden students' understanding of opportunities that exist in the computing field.

(Trevor Nelligan, Raniero Lara-Garduno, Paul Taele, Seth Polsley)

The SRL members began the invited talk with doctoral student Taele first introducing the lab and the present lab members. First-year Master's student Polsley then proceeded to present on Mechanix, an SRL project on an intelligent sketch educational application that is designed to assist civil and mechanical engineering students in learning about foundational engineering concepts of trusses and free-body diagrams from the application's automatic interpretation, understanding, and critique of the students' hand-drawn diagrams.

With Mechanix, a student can using a touch-enabled tablet or touchscreen monitor to draw their homework solutions, and Polsley and fellow Master's student Turner drew a truss example to demonstrate Mechanix's ability to automatically recognize, label, and critique the sketched input. In addition to drawing diagrams such as trusses and free-body diagrams, engineering students can also use Mechanix to input forces in any direction and subsequently calculate the value of any of its variables. Ultimately, the motivation behind the application is to enable engineering students to receive instantaneous critique and grading from completing statics homework exercises step-by-step, since students lack such personalized feedback on homeworks done on pen-and-paper for classes ranging from a hundred to a thousand students.

(The Mechanix software system)

Turner followed up the presentation on Mechanix by describing the challenges that in exist in creating such a system, including how since everyone draws in different styles and habits, the application would need to able to recognize various sketched figures regardless of drawing behavior or order. Additionally, such an application would need to be able to recognize the sketch as the student intended, and does so by using perception-based algorithms such as geometric primitives recognizer PaleoSketch that was developed by the Sketch Recognition Lab.
Polsley and Turner transitioned the presentation from Mechanix to one of SRL's more recent research work in PerSketchTivity for engineering sketching. The application enables students to gain experience and practice in three-dimensional drawing by first guiding students in focusing on building muscle memory of two-dimensional shapes like squares, lines, circles, and ellipses, and then teaches them perspective drawing using vanishing points, since both muscle memory and perspective drawing skills are crucial for improving engineering mastery, constructing blueprints, and so on. The PerSketchTivity project is a collaboration project between SRL and researchers Dr. Julie Linsey, Dr. Wayne Li, and Master's student Blake Williford.

(The Persketchtivity software system)

The SRL presentation shifted focus away from engineering sketching, with doctoral student Taele introducing SRL research projects that target other domains. Two projects that Taele briefly touched upon was prior and on-going research work in recognizing sketches made from scratching sounds made on physical surfaces and predicting the age level of a sketcher based on handwritten symbol input on tablets, respectively. Taele then transitioned to language sketching projects such as Hashigo that focus on assisting students in learning written symbols and characters in East Asian language symbols such as Chinese, Japanese, and Korean. Similar to the engineering sketch projects, students are able to receive feedback on the visual correctness of their sketched inputs, but the language sketch projects also focus on writing elements such as stroke order and direction that is important in East Asian languages.

Turner continued off from Taele's presentation on SRL's language sketch to briefly introduce another of SRL's on-going project called CourseSketch, a course management system for sketch recognition-based assignments similar to Mechanix but that generalizes to a variety of courses. One significant motivation is to enable students to go through homework exercises that require sketching and then receive instantaneous feedback. CourseSketch is currently employed in a philosophy course called Introduction to Logic at Texas A&M University to provide students with practice over drawing logic proofs.

Master's student Powell completed the invited talk with his presentation of SRL haptic research projects by first briefly describing GeoTrooper, a prior work that assists army paratroopers in navigating to waypoints using haptic feedback. Powell then described another SRL haptic research project called HaptiMoto that provides similar functionality for motorcycle navigation, which was supplemented by showcasing Discovery Daily Planet video segment (http://www..discovery.ca/Video?vid=531190) highlighting both GeoTrooper and HaptiMoto. Following the video showcase, Powell demonstrated one of the haptic vest prototypes to the audience, which uses vibrotactile sensors and a Bluetooth receiver for hands-free GPS navigation, and described the vest's usage and the progression of the vest's creation and design. Powell described the vest's navigation capabilities as originally being handled by mobile device-input setup, but were moved to a hands-free system via Bluetooth and vibrations provided on either or both shoulders on the vest for safer navigating.

With the end of Powell's presentation, the SRL members successfully completed their invited talk to the TACS audience on SRL's collection of prior and on-going research projects, and hoped that the undergraduate students gained better insight of the types of exciting research opportunities that are available to them and that take advantage of the knowledge that they learned from their computer science coursework.

(Cassandra Oduola, Trevor Nelligan, David Turner, Seth Polsley, Raniero Lara-Garduno)