Sketch Recognition Lab Summer Defense Follow-Ups

Over the summer of 2016, three Sketch Recognition Lab members, Siddartha Karthik, Swarna Keshavabhotla, and Stephanie Valentine, successfully defended their theses and dissertation in front of their committees and Director Tracy Hammond. The back-to-back-to-back defenses made for a long day on June 10th, but one that was well worth the work and the hours put in by everyone involved.

(Siddartha Karthik)

Siddartha Karthik, who is getting his MS in Computer Science and is set to graduate in August 2016, presented his thesis work on data mining and labeling in sketch recognition. His advisor was Dr. Hammond. Motivated by his want “to do some kind of data mining and machine learning” and a sketch recognition class, which he greatly enjoyed, that he took when he was in his first semester at Texas A&M University, Karthik focused his research on the recognition and labeling of images. As stated in his abstract:

“Recognizing sketches instantaneously is necessary to build beautiful interfaces with real time feedback. There are various techniques to quickly recognize sketches into ten or twenty classes. However for much larger datasets of sketches from a large number of classes, these existing techniques can take an extended period of time to accurately classify an incoming sketch and require significant computational overhead. Thus, to make classification of large datasets feasible, we propose using multiple stages of recognition… This process both significantly reduces the time taken to classify such huge datasets of sketches, and increases both the accuracy and precision of the recognition.”

Now that his thesis has been defended and he’s set to graduate at the end of the summer, Karthik is preparing to start working at Uber in San Francisco. When asked about where he saw his research going, he replied, “The first thing I have in my mind is to get it published. After that, there are a lot of ways in which others could expand on my work.”

Furthermore, if he could change one thing about his time in SRL, Karthik stated that he “Should have started working on [his] research a couple of semesters earlier” to make the process easier and less stressful.

(Stephanie Valentine defending her dissertation)

In a much different area of sketch recognition, Stephanie Valentine defended her dissertation and became Dr. Valentine with her research about children's social networking. Valentine has worked for years developing KidGab, a social media site designed for children under the age of 13, and promoting her nonprofit, Wired Youth.

As Valentine stated, "Lots of people have studied teens and adults, but children's social networking behaviors (from the perspective of the social network designer/admin) had never been published. I looked at questions like, What do pre-adolescents really talk about online? Are they likely to conform to one another? Which activities and affordances engage them most?"

When asked about her motivation for the research, Valentine responded that she wanted her work to be meaningful and good for humanity. Knowing the pain of cyberbullying, she saw plenty of research regarding cyberbully, but most was focused on reporting it or detecting it, rather than extinguishing it.

"I chose work which simultaneously prevents cyberbullying and is meaningful to the research world," she commented. "Kids are going to be online anyway, so why not give them a safe space to learn how to social network before they are expected to grasp the concepts and safety procedures of adult networks it at the level of an adult? My social network, KidGab; my nonprofit, Wired Youth; and now the iPhone and Android apps are all dedicated to that aim."

Valentine plans to stay in SRL as a research assistant and to continue her work on KidGab and has received a grant from the National Science Foundation which will fund her research for about a year. She plans to have a graduate assistant for research as she works to build her social network site even more, reaching more and more children every day.

"This lab has been amazing in so many ways," she reflected when asked about her takeaways. "When I arrived, the lab members nicknamed it the "rainbow lab" because we had people from such diverse origins working together in collective awesomeness. Dr. Hammond has a knack for recruiting very personable and energetic students, so the lab is frequently abuzz with some new exciting thing or another (be it technological, social, algorithmic, Pokemon-related, or anything else)."

Continuing by saying that Dr. Hammond has been supportive beyond the call of duty for an adviser, Valentine's "biggest takeway will be in [her] interactions and observations of [Dr. Hammond]." She continued, "I want to be her when I grow up: a scientist with a wide lens of knowledge, the life of every conversation and the brightest presence in any room. And now she's doing all that with a baby in her arms. It's unbelievable."

When asked about what direction her research is going, Valentine answered that it could go in so many ways that she often has to stop herself from asking more questions about it. Be it studying digital conformity found in children online or the creative processes of creation that children go through when allowed to draw and post their own sketches, the path which KidGab's research could go is endless and in an area of study that is relatively untapped. 

(Valentine explaining KidGab to her audience)

Upon reflecting on her time at SRL and her dissertation research, Valentine isn't sure if she would go back and change something, if she could, though she would love to go back and converse with her past-self.

"I would tell myself that the next few years are going to be hard. Your work won't always be respected because of its lack of theoretical algorithmic-ey-ness. Your thesis won't look like a Greek mathematician's notebook, and that's okay. People will reason that dealing with math and computers is much harder than dealing with humans, right? And you're not just dealing with humans, you're dealing with children. Piece of cake, they'll say. False. So false. They've clearly never tried to extract scientific meaning from a child's digital poetry. I would tell myself that you'll have to fight and fight some more for what you believe in, forging your own path when there isn't one before you. You'll find a way.

Not everyone will be happy, computer scientist reviewers can be mean and unreasonable, so take solace in those who do believe in your work. There are awesome people who will help and guide you. Pay attention to them. But in the end, you'll be so proud of your dissertation -- your legacy -- and the difference you made in the hearts of many a Girl Scout!"

Everyone in the Sketch Recognition Lab would like to congratulate Siddartha Karthik, Swarna Keshavabhotla, and Dr. Stephanie Valentine on all of their hard work. We are so proud of everything they have accomplished - and will accomplish in the future.

SRL MS Thesis Defense of Swarna Keshavabhotla. Friday, June 10. Title: PerSketchTivity: Recognition and Progressive Learning Analysis

Thesis Defense

Friday, June 10

Title: PerSketchTivity: Recognition and Progressive Learning Analysis

Swarna Keshavabhotla

3:30pm Friday, June 10, 2016

Room 326 Teague Building

Abstract

Development of novice computing technologies has opened the door for computer- based tutoring systems in areas where it was previously not possible. Sketch-based tutoring systems are being developed for subject domains, which involve diagram- matic representations for problem solving. In this thesis, we present PerSketchTivity, a sketch-based tutoring system for design sketching that allows students to hone their skills in design sketching and self-regulated learning through real-time feedback. Students learn design-sketching fundamentals through drawing exercises of reference shapes starting from basic to complex shapes in all dimensions and subsequently receive real-time feedback assessing their performance.

PerSketchTivity consists of a recognition system, which recognizes the correctness of the sketch on the fly, on completion, feedback system provides real-time feedback and the evaluation system evaluates the sketch based on various features like error, smoothness and speed. The focus of this thesis is to evaluate the performance of the system in terms of the recognition efficiency and also the impact on the drawing skills of the students practicing with this system.

Biography

Swarna Keshavabhotla is a MS candidate at Texas A&M University department of Computer Science and Engineering. She is a member of Dr. Tracy Hammond's Sketch Recognition Lab. The focus of her research is development of sketch recognition algorithms for assessment in Intelligent Tutoring Systems and feedback mechanisms for enhancing the impact of the system on students' skills. Swarna received her Bachelors degree in India.

Advisor: Dr. Tracy Hammond

SRL MS Thesis Defense of Siddhartha Karthik Copesetty. Friday, June 10. Title: Labeling by Example

Thesis Defense

Friday, June 10

 

Title: Labeling by Example

Siddhartha Karthik Copesetty

1:00pm Friday, June 10, 2016

Room 326 Teague Building

Abstract

Sketch Recognition is recognition of hand drawn diagrams. Recognizing sketches instantaneously, is necessary to build beautiful interfaces with real time feedback. There are various techniques to quickly recognize sketches into ten or even twenty classes. But, what if we have 100,000 sketches and want to classify them into 3000 different classes? Using the existing techniques, it takes forever and ever to accurately classify an incoming sketch into one of these 3000 classes. For example, a class of hundred sketches takes two hours to get classified into one of the 3000 classes. This is very very slow, takes significant computation overhead and is not practical. So, to make things faster, we propose to have multiple stages of recognition. In the initial stage, the sketch is recognized starting from the outer level, moving level by level into the center of the sketch. This recognition is done by matching it against a set of sketch domain descriptions, resulting in a list of classes that the sketch could possible be, along with the accuracy and precision for each. For the ones with accuracy less than a threshold value, they go through a second stage of recognition. In this stage, feature values are calculated and evaluated against our model to accurately classify the sketch. Thus, the time taken to classify such huge datasets of sketches decreases significantly with increase in accuracy and precision.

Biography

Siddhartha Karthik Copesetty is a master’s student in the Sketch Recognition Lab. He completed his undergraduate degree in Computer Science at National Institute of Technology, Tiruchirappalli, India. He was a software engineering intern at Yahoo!, Sunnyvale last summer.

Advisor: Dr. Tracy Hammond

SRL PhD Dissertation defense Stephanie Valentine. Friday, June 10. Title: Design, Deployment, Identity, & Conformity: An Analysis of Children's Online Social Networks

Dissertation Defense

Friday, June 10

Title: Design, Deployment, Identity, & Conformity: An Analysis of Children's Online Social Networks

Stephanie Valentine

10:00am Friday, June 10, 2016

Room 326 Teague Building

Abstract

Preadolescents (children aged 7 to 12 years) are participating on online social networks whether we, as a society, like it or not. Enacted by the United States Congress in 1998, the collection of online data about children under the age of 13 is illegal without express parental consent. As such, most mainstream social networks such as Twitter, Facebook, and Instagram limit their registration by requiring new users to assert that they are at least 13 years of age, an assertion which is often falsified. Researchers, bound by the same legal requirements regarding online data collection, have resorted to surveys and interviews to understand how and why children interact on social networks. While valuable, these prior works explain only what children say they do online, and not what they actually do on a daily basis. In this work, we describe the design, development, deployment, and analysis of our own online social network for children, KidGab. This work explores common social networking affordances for adults and their suitability for child audiences; analyzes the participatory behaviors of our users (Girl Scouts from around central Texas) and describes how they shaped KidGab's continuing growth; discusses our quantitative analysis of users' tendencies and proclivities toward identity exploration; leverages graph algorithms and link-analysis techniques to understand the sociality of conformity on the network; and finally, this work describes the lessons we learned about children's social networks and social networking througout KidGab's 450 days of active deployment. 

Biography

Stephanie Valentine is a PhD candidate in the Department of Computer Science & Engineering at Texas A&M University. A Nebraska native, Valentine completed a BA in Computer Science with a minor in Electronic Publishing from Saint Mary's University of Minnesota. Valentine is an NSF Graduate Fellow, winner of the Susan M. Arseven ’75 Make A-Difference Award, and Vice President of the CSE Departmental graduate student association. Valentine's research focuses around understanding how children communicate in online social networks and empowering children to have safe, healthy, and expressive digital friendships. Valentine is also the founding president of Wired Youth, Inc., a 501(c)3 non-profit organization that works to educate the community about safe social networking for children as an active prevention strategy for cyberbullying, online predation, and other cyberthreats.Stephanie Valentine is a PhD student in the Department of Computer Science & Engineering at Texas A&M University. A Nebraska native, Valentine completed a BA in Computer Science with a minor in Electronic Publishing from Saint Mary's University of Minnesota. Valentine is an NSF Graduate Fellow, winner of the Susan M. Arseven ’75 Make-A-Difference Award, recipient of the 2016 NCWIT Collegiate Award (Honorable Mention), and winner of the 2015 Texas A&M University Department of Computer Science & Engineering Mentoring Excellence Award. Valentine's research focuses around understanding how children communicate on online social networks and empowering children to have safe, healthy, and expressive digital friendships. Valentine is also the founding president of Wired Youth, Inc., a 501(c)3 non-profit organization that works to educate the community about safe social networking for children as an active prevention strategy for cyberbullying, online predation, and other cyberthreats.

Advisor: Dr. Tracy Hammond

SRL at Aggieland Saturday

On February 13th, the Sketch Recognition Lab volunteered to participate in Aggieland Saturday. This event is a campus-wide open-house for visitors, primarily high school students and their families, to come and see the happenings in various departments across the university. Members of SRL, directed by Dr. Tracy Hammond, signed up to fill presentation and lab tour slots to demonstrate a few projects and to talk to the students and families who came to visit.

The main Texas A&M University campus was bustling with visitors, tour guides, volunteers, and various organizations and businesses from around town. Some of the students that came to the SRL presentation were from across the state, having made the drive to come to the event and to explore what TAMU had to offer.

Over the span of the day, SRL gave four demonstrations for visitors who were welcomed to Aggieland and the lab. Seth Polsley lead the presentation of projects.

(Left to Right: Cassandara Oduola, Seth Polsley, and Pernendu Kaul presenting to visitors)

In their last presenation of the day, SRL members Cassandara Oduola, Seth Polsley, and Pernendu Kaul ran demonstrations of eye tracking technology, Mechanix, and Persketchitivity.

Polsley, in his welcome speech, gave a brief introduction of a few projects to demonstrate the vast areas of interest that are covered in the lab - from KidGab, a social media for children to learn about Internet safety, all the way to SmartStrokes, a digitalized version of neuropsychology tests in the medical field.

"All kinds of things go on in our lab. We have lots of grad students and lots and lots of undergrad students," he continued. "So I'm so happy that you guys are here and get to see what's going on in the lab."

With the greeting and quick summary of the lab complete, the volunteers moved on to demonstrations of a few projects they set up for the visitors to interact with.

Kaul kicked this portion of the presentation with an eye tracking system. While it was not hooked up to a program to use it with, a visitor was able to calibrate the system and see as the screen reacted - a series of eight circles with plus signs inside of them were arranged around the perimeter of the screen, a ninth circle was in the middle of the screen. After calibration, the circle a user stares at will light up red.

"Is it going to hurt?" The visitor questioned when he sat in front of the mechanism. Kaul quickly assured him that it wouldn't and not to worry.

"We would need you to sign a waiver if it did," Oduola joked, easing the tension of the question.

"When the documents come out, that's when you want to hesitate," Polsley teased further before continuing more seriously. "Eye tracking is actually very interesting because it's very non-invasive and it's becoming available for the desktop. We don't actually have any of the projects that use eye tracking set up today, because most of the people who use it aren't here, but you can see the system."

(Kaul and a volunteer demonstrating the eye tracking technology)

The visiting high school students eagerly watched as the system worked, highlighting each dot that was being traced in various patterns.

The demonstration moved onward and onto the large Microsoft Surface. Polsley opened Mechanix on the large touchscreen, describing that the program was made for teaching civil engineering students how to solve truss and free diagram problems.

"It's primarily to help educators in large classes automatically have some evaluation metric for their students," he explained. "And then for the students to get feedback as they go through these problems."

He took a few minutes to explain the interface and invite the visitors to interact with the large touchscreen by sketching and solving the tutorial problems presented by Mechanix.

As they continued to explain the program, Polsley took the visitors step by step through a full, but simple, practice problem. For each part, he asked for someone to draw and test the program as he described what was happening in the system to create the feedback and the images that appeared on the screen.

“This is a collaborative way of solving a problem,” he joked as he watched someone draw a force direction pointing to a node in a truss. “This system is built with multiple layers. There is the sketch recognition aspect and error recognizers, but we also have a linear algebra solver that can understand these equations, so that, as students are going through the homework, we can evaluate if they’re solving every part of the problem correctly. So this is quite an expansive system.”

After concluding their segment on Mechanix, the SRL volunteers moved on to demonstrating a similar program, Persketchativity, a program that is designed to help people learn drawing. It is particularly useful for people doing graphic design or art courses in which students need to learn to draw perspective. As with Mechanix, Persketchativity starts users off slowly with practices regarding drawing lines and simple geometric shapes. Eventually, its lessons get more and more complex until they involve drawing perspective and more advanced shapes.

(SRL Members Seth Polsley and Cassandara Oduola demonstrating Persketchativity)

Unique to Persketchativity is the error analysis, which shows a graphical image of how far off mark a drawing is by connecting the input sketch with the correct lines. 

Polsley quickly took the visitors through the first lesson on lines so that he could demonstrate the analytics of the program. “So at the end it’s going to give me some metrics on my speed, my smoothness, my error analysis, and things like that. Anything a professor might be interested in.” He opened another lesson as he continued. “So you can see how this would be a helpful tool for students and the metrics at the end make it nice for the professor to use.”

After the demonstration concluded, the SRL volunteers stressed that, while education was a large area that held opportunities for the lab, the projects created and research studied spanned a wide variety of fields. They touched on, for example, how there was also a focus on health research in the lab, because there are a vast number of ways to use the recognition systems used in education to help in medical fields.

“We have a lot of projects,” Polsley commented as his fellow lab members agreed with nods. “It’s a big lab. We’ve made navigation systems using haptic, which is just touch-vibration. We have a vest that was developed for the military to help troopers find rendezvous points and a group that is working on a similar system to help the visually impaired navigate based on touch.”

After the SRL members concluded their speech, they opened the floor to questions and interactions with the visiting students and their parents. Many of the students expressed enthusiasm for what they were shown and finding ways to be involved with the lab, regardless of whether or not they were interested in computer science or other branches of technology or engineering.

“This is a huge research institution,” Polsley offered as both a final statement and an invitation for future interaction with the lab. “It’s a massive university, and has a lot of potential for anything that interests you, so it’s all pretty exciting.”

SRL wins 5 awards at the 2015 Spring IAP Poster Competition and CSE Banquet

On April 20th, 2015, the Sketch Recognition Lab, directed by Dr. Tracy Hammond, kicked off a busy, bustling afternoon that included both the IAP Poster Competition and, soon after, the Computer Science and Engineering Departmental Banquet and Awards Ceremony. 

Seven members of SRL, six graduates and one undergraduate presented posters at the IAP Competition. Many of the judges later commented on how impressed they were with the work and presentations of the SRL members.

The hard work that everyone put into their presentations was reflected at the awards ceremony later that evening, with Seth Polsley winning second place and Stephanie Valentine tying for first place in the IAP poster competition!

Additionally, Trevor Nelligan won the Undergraduate Research Award, Stephanie Valentine won the Mentorship Excellence Award, and Rafael (Rafa) Moreno won the Undergraduate Leadership Award at the CSE banquet! Congratulations to everyone!! 

Rafa Moreno, who recently joined SRL as an undergraduate researcher working on Smart Strokes with Raniero Lara Garduno, stated, "It feels pretty good to win the award, not for the award itself, but the fact that someone agrees with you on the impact of what you are doing, and how many people it is positively affecting."

Outside of the lab, Moreno is interested in machine learning and computer vision as it relates to artificial intelligence. He is also the president of TAMUHack and helps the organization and growth of the "hackathon".

SRL MS Student Seth Polsley, one of six graduate students who presented in the poster competition, won second place for his research with Mechanix.

Polsley's poster focused on the trends regarding student interaction with the problems given to them in Mechanix. For example, how many times have students tried to correctly answer specific problems? At what point in an assignment did a concept throw students off and caused them to struggle?

Everyone has experienced the stress and frustration that occurs when a problem can’t be solved immediately or even after a few tries. The more frustration that builds up, the harder and more irritating the problem becomes. However, sometimes the best solution is to let it go. Relax. Coming back to a problem an hour or a day later can sometimes make it easier to think logically and solve it. 

“Sometimes you need a break,” Polsley stated about struggling with a question. “Maybe we can add something in Mechanix that pops up after a certain number of attempts on a question. It may just be something like ‘go ahead and continue and come back later.’” 

Like Polsley, Nelligan's research and studies have focused on Mechanix this semester. Trevor Nelligan won the Undergraduate Research Excellence Award for his presentation for the IAP competition.

Mechanix is an innovative and digital learning area for beginners. The program itself allows for instructors to create practice problems for students. Users can then sketch images into the program and add force and force direction to solve the problems and produce formulas.

Once a problem is worked out by a student, they can submit it for instant feedback and correction. This is particularly helpful in a large class where handwritten assignments are necessary – for example, an engineering or mathematics class. While it would take time for an instructor to grade and comment on every student’s assignment, Mechanix allows for students to be automatically evaluated and given feedback to help students understand what was missed in the problem.

SRL PhD Candidate Stephanie Valentine also presented at the IAP Competition and won first place for her research covering her work with KidGab. Valentine’s colorful poster resembled a screenshot of a social media site with the flair of her own touch. 

Along with her poster, Valentine set up a tablet in her presentation that constantly showed a looped video about how sketches on KidGab can become videos of the drawing being created, pen stroke by pen stroke. While this is currently a fun, leisure application found in KidGab, it can be used in an educational setting to make videos about concepts learned in class – for example, a sketch about how to solve a problem using long division.

In her presentation, Valentine gave a brief rundown on her social network for kids aged 7 to 12. Her goal of educating children on how to be safe online and have a responsible digital self were a driving force behind her speech. She also touched on how KidGab is used by children that she has worked with during Digital Friendship workshops to identify and express themselves.

  

Congratulations to everyone who participated and to everyone who was awarded for their posters and involvement with the CSE Department!

With the stress of research competitions out of the way for the moment, SRL attended the end of the year banquet for the Computer Science and Engineering Department at Texas A&M University. The senior capstone groups – who have mentors in the SRL – had posters on display for the first half of the night.

With a beautiful layout and delicious food, the evening was a nice breather to the hectic schedule that members of the SRL usually face. In fact, it became an event for everyone to have fun, chat, and shake the stress off on the dance floor – perhaps while learning a few new steps along the way.

SRL Dissertation Defense Hong-Hoe (Ayden) Kim Friday, March 4, 2016 , Title: A Fine Motor Skill Classifying Framework to Support Children's Self-regulation Skills and School Readiness

SRL Dissertation Defense

Hong-Hoe (Ayden) Kim

Friday, March 4, 2016

Title:  A Fine Motor Skill Classifying Framework to Support Children's Self-regulation Skills and School Readiness

Hong-hoe Kim

3:30pm Friday, March 4, 2016

Room 323 Teague Building

Abstract

Children's self-regulation skills predict their school-readiness and social behaviors, and assessing these skills enables parents to target areas for improvement children to enter school ready to learn and achieve. To assess children's fine motor skills, current educators are assessing those skills by either determining their shape drawing correctness or measuring their drawing time durations through paper-based assessments. However, the methods involve human experts manually assessing children's fine motor skills, which are time consuming and prone to human error and bias. We introduce our fine motor skill classifying framework based on children's digital drawings on tablet-computers. The framework contains two fine motor skill classifiers and a sketch-based educational interface.

Biography

Hong-hoe Kim is a PhD Candidate in sketch-recognition lab at Texas A&M University under the supervision of Dr. Tracy Hammond. He received his Masters degree in Computer Science from Texas A&M University and B.S. degree in Computer Science from Soongsil University in Korea. His research area includes Child-Computer Interaction, Human-Computer Interaction (HCI), Machine Learning, and Educational Psychology.

Advisor: Dr. Tracy Hammond

SRL MS Thesis Defense: Purnendu Kaul, March 3, 2016, Gaze Assisted Classification of OnScreen Tasks and User Activities

SRL Thesis Defense:
Purnendu Kaul
Thursday, March 3

Title:  Gaze Assisted Classification of On-Screen Tasks (by Difficulty Level) and User Activities (Reading, Writing/Typing, Image-Gazing)
Purnendu Kaul
12:30pm Thursday, March 3, 2016
323 Teague, CSE, TAMU

Abstract
Intelligent tutoring systems(ITS) are commonly used to indirectly assist classroom instructors by helping them deliver the learning material and assess students' progress as they learn. Today, such systems put the onus of asking for appropriate help on students, instead of assessing their needs automatically. This provides an opportunity to make systems which are capable of adapting to the cognitive states of students as they learn.

We have shown that Gaze-assisted human-computer interaction is a means of transforming these Intelligent tutoring systems (ITS) into more proactive and adaptive systems. A system with eye tracking capability can be trained to learn cognitive states of a user and offer contextual assistance. In this research, we conducted an experiment using Mechanix, a sketch based ITS system, that helps students learn how to solve truss problems.

Through this experiment, we investigated the possibility of using eye gaze data to classify problems being solved by students as difficult, medium, or hard. We also classify the activity being performed by users as "reading", "gazing at an image," and "drawing/typing." We only used those gaze features which can be calculated in real time, and are not dependent on the duration of activity on the system.  The results show that gaze features can clearly differentiate between the activities with an accuracy of 94%, and classify the problems as easy, medium, or hard with an accuracy of more than 70%.

Biography
Purnendu is a masters student in the Sketch recognition lab. He completed his undergraduate degree at National Institute of Technology Kurukshetra, India and worked at the Indian Institute of Technology for a year before starting the graduate program at Texas A&M University. He was a summer software intern at Schlumberger Information Solutions in Houston during the summer of 2014.

Advisor: Dr. Tracy Hammond

SRL Thesis Defense: Shalini Priya Ashok Kumar Thursday, March 3 Title: Evaluation of Conceptual Sketches on Stylus-based Devices

SRL Thesis Defense: Shalini Priya Ashok Kumar
Thursday, March 3

Title:  Evaluation of Conceptual Sketches on Stylus-based Devices

Shalini Priya Ashok Kumar
9:00am Thursday, March 3, 2016
Room  323 Teague Building

Abstract
Design Sketching is an important tool for designers and creative professionals to express their ideas and thoughts onto visual medium. Being a very critical and versatile skill for engineering students, this course is often taught in universities on pen and paper. However, this traditional pedagogy is limited by the availability of human instructors for their feedback. Using intelligent interfaces this problem can be solved where we try to mimic the feedback given by an instructor and assess the student drawn sketches to give them insight of the areas they need to improve on. PerSketchTivity is an intelligent tutoring system which allows students to practice their drawing fundamentals and gives them real-time assessment and feedback. This research deals with coming up with the grading rubric that will enable us to grade students from their sketch data.
Biography
Shalini Priya Ashok Kumar got her Bachelor of Technology degree in Information Technology from National Institute of Technology Karnataka, Surathkal. She worked for Citrix R&D India, Bangalore for a couple of years after which she joined Texas A&M University in the Masters program. She works in the Sketch Recognition Lab on developing software for Design sketching students. Her interests are Artificial Intelligence, Machine learning and Human Computer Interaction.

SRL Dissertation Defense: Folami Alamudun Monday, February 29 Title: Analysis of Visuo-cognitive Behavior in Screening Mammography

SRL Dissertation Defense:
Folami Alamudun
Monday, February 29

Title:  Analysis of Visuo-cognitive Behavior in Screening Mammography

Folami Alamudun
12:00pm Monday, February 29, 2016
Room 323 Teague Building

Abstract
Improved precision in modeling and predicting human behavior and the underlying metacognitive processes is now possible thanks to significant advances in bio-sensing device technology and improved technique in machine intelligence.  Eye tracking bio-sensors measure psycho-physiological response through changes in configuration of the human eye. These changes include positional measures such as visual fixation, saccadic movements, and scanpath, and non-positional measures such as blinks and pupil dilation and constriction. Using data from eye-tracking sensors, we can model human perception, cognitive processes, and responses to external stimuli.

In this study, we investigate visuo-cognitive behavior in screening mammography under clinically equivalent experimental conditions. We examined the behavior of 10 image readers (three breast-imaging radiologists and seven Radiology residents) during the diagnostic decision process for breast cancer in screening mammography. Using a head-mounted eye tracking device, we recorded eye movements, pupil response, and diagnostic decisions from each image reader for 100 screening mammograms. Our corpus of mammograms comprised cases of varied pathology and breast parenchyma density.

We proposed algorithms for extraction of primitives, which encode discriminative patterns in positional and non-positional measures of the eye. These primitives capture changes correlate with individual radiologists, radiologists’ experience level, case pathology, breast parenchyma density, and diagnostic decision. We evaluated the effectiveness of these primitives through performance measures using ten-fold cross-validation for training and testing a simple learning algorithm.

Our results suggest that a combination of machine intelligence and new bio-sensing modalities is an adequate predictor for the characteristics of a mammographic case and image readers’ diagnostic performance. Our results also suggest that primitives characterizing eye movements can be useful for biometric identification of radiologists. These findings are impactful in real-time performance monitoring and personalized intelligent training and evaluation systems in screening mammography.

Biography
Folami Alamudun is a doctoral candidate in the Department of Computer Science and Engineering at Texas A&M University working directly with Dr. Tracy Hammond in the Sketch Recognition Laboratory (SRL) and in collaboration with Dr. Georgia Tourassi at the Oak Ridge National laboratory's (ORNL) Biomedical Sciences and Engineering Center.

His research focuses on machine intelligence and bio-sensing device applications in user behavior modeling.

Advisor: Dr. Tracy Hammond

KidGab at 2016 Girl Scout STEMfest

On February 6, Texas A&M University hosted the Girl Scouts of Central Texas' 2016 STEMfest. Spanning across the main lobby and hall of first floor of the Jack E. Brown Building, organizations and student researchers set up booths to present their projects and enthusiasm for science, technology, engineering, and mathematical fields of study.

Stephanie Valentine attended to represent the Sketch Recognition Lab, directed by Dr. Tracy Hammond, and her project, KidGab. Her booth, with its bright colors and bright smiles, attracted almost every girl that passed by. Parents and children alike showed interest in KidGab and the Digital Friendship Program.

KidGab is Valentine's social media platform that is made specifically for Girl Scouts. It is a safe environment for girls to learn how to be safe online and to stay connected with other scouts. Parental involvement is needed to register for an account and to keep the account active over time. Valentine's Digital Friendship Program is an extension of her non-profit organization, Wired Youth.

"Every time I host events, the girls always ask me 'when do we get to meet the Aggies?'" Emily Magnotta, the Program and Travel Specialist of the Girl Scouts of Central Texas, commented about the scouts excitement about the event in her opening talk. She went on to state that she, too, looked forward to STEMfest, jokingly stating, “I get to learn a while new bunch of stuff - like that there is more than one type of engineers.”

This year, the event was partnered with the Society of Women Engineers. Diverse organizations and departments from across campus came out to interact with the girls. Ranging from Agriculture, Aggieloop, and Dell computers all the way to the Society of Test Flight Engineers, outreach projects in engineering, and the International Society of Black Engineers, there was something there for everyone. About 80 girl scouts and their parents attended from around central Texas, some even making a six hour drive to participate in it. There was a large age range in the girls - the youngest being 5-years-old and the eldest being 17.

The motivation and purpose behind STEMfest is to inspire girls to become interested in STEM-related areas of study and careers. As Magnotta stated in her opening speech, only 1 in 5 people in related careers are women, and it's important for girls not only to be interested in the subjects taught in school, but to also know that STEM careers are an option and understand that they are capable of being involved in STEM.

This motivation was exemplified in pop culture reference when Magnotta had Valentine retell the story of the engineering-themed Barbie coloring book. In the book, two male characters are working on a project. When Barbie walks in and sees what they are doing, she states that she could never do what they do because it was too difficult.

Later that day, one girl was doing a scavenger hunt - a list of questions that she needed answered by the student-presenters - and asked Valentine what programs that engineers used for coding. Instead of just answering, Valentine pulled out her laptop and showed her the codes she had just finished updating for KidGab. She took the girl through a step-by-step, visual tour of coding.

(Valentine talking with a Girl Scout about coding)

When she was done, Valentine asked the girl scout if she knew who coded all the things on the screen before her. When the girl shook her head, Valentine pointed to herself with a smile, "I did." Then she asked the girl if, one day, she thought that she could program something, too. When the girl shook her head, Valentine responded, "why not? I did this. You can, too."

Over the course of the afternoon, girl scouts came to Valentine's booth to talk with her about her project and play with the avatar creation that was set up on four iPads. The scouts loved the interactivity and being able to create avatars, the parents liked how safe and rooted it was.

"It's really fun," one of the younger scouts said while posting a drawing on one of the demo accounts' wall.

Romain Cambier and Marc Honore Reflect on Thier Time at SRL

During spring 2015 and the first part of summer, the Sketch Recognition Lab and director Dr. Tracy Hammond happily hosted Marc Honore and Romain Cambier, two students studying abroad from their homes in Belgium.

This, however, was not their first trip to Texas A&M University. A few years ago, they won a week trip as a first place award of a challenge that they won in Europe.

“It was a really cool stay – we really enjoyed what we saw here,” Cambier explained their motivation for spending spring in Texas. “When we decided to work on a master thesis, we just talked about A&M. It was a great experience, so why not try again for a longer period?”

While at SRL, Cambier and Honore combined their interest of computer intel and science to work together on a prototype for a project titled SmartBricks. Consider, for a moment, Legos that could be used to build functioning, programmable technology. That’s SmartBricks. With and easy-to-use code creator (that looks a lot like a digital drag-and-drop circuit board), users can program individual blocks to do various functions in a process.

(Marc Honore [left] and Romain Cambier [right] working on SmartBricks)

In the video here, SmartBricks is seen in action. In the video, it is programmed to be a screen cleaner for a smartphone. Each command is shown in order throughout the process.

When asked if they would take the chance to return to SRL, Cambier and Honore smiled.

“A lot of people have been asking us to stay,” Honore stated.

“We really enjoyed our stay here, really enjoyed the people – the people were all very welcoming,” Cambier added before both quietly agreed that they would probably take the opportunity to return, if it was presented to them.

Being a part of Texas A&M University and SRL has given Cambier and Honore many experiences, both good and challenging.

It only took Honore a beat to list the weekly lab meetings as one of his favorite memories and aspects of the lab.

“It means there is a lot of interaction between all of us, which I think is really good for SRL,” he stated. “And all the activities that are done outside the lab – like our waffle day.”

(Stephanie Valentine [left], Marc Honore [center, back], and Raniero Lara-Garduno [right] at Waffle Day 2015)

Waffle Day 2015 is a day that will live on forever in SRL memory. The event was announced via an email titled “Belgian waffle day.” One morning, Cambier and Honore decided to bake waffles for the lab. Like a wildfire, the excitement to eat Belgium waffles made by Belgians spread. The event was so popular (and the cooks were so good) that there was a second event a month after the first.

“Everyone is very friendly,” Cambier agreed. “It’s very welcoming and very cool.”

Of course, both students encountered a few obstacles while studying abroad. Most of which occurred before they ever arrived. Obtaining a student visa from the federal government is not a cake walk. Papers, forms, and background checks slow the process down exponentially. It took about three weeks for Cambier and Honore to complete the process.

“One thing that was difficult, here, during the first week is to find everything, from buildings to places to eat,” Cambier added about Texas A&M University.

However, no challenge could keep them away from having fun and touring a few Texas hotspots, such as San Antonio (which was Honore’s favorite place visited), Austin, and Houston (which was combined with the Space Center to create Cambier’s favorite visit).

(Romain Cambier working in SRL)

In a word, their departure from SRL was bittersweet. Everyone was happy to have made two new friends, and were sad to see them go so soon. Although Cambier and Honore were leaving TAMU, they were going home to all the things they were excited to see again:

“My family,” Honore instantly listed.

“Girlfriend,” Cambier commented, getting a quick agreement from Honore.

“The food.”

“Oh, yeah! The food!”

As their visit came to a close, Cambier and Honore gave a few tidbits of advice based on their hindsight.

“The most difficult part [of transitioning into the lab] was finding a desk,” Honore joked. “But besides that, the first day we came here was a Monday, so there was a lab meeting and we got to meet a lot of people. Then, afterward, Paul and Jung-In took us out to lunch. After that, it just took us a while to get all the stuff we needed.”

“Honestly, people were so welcoming and friendly that it was easy to discuss and find what we needed,” Cambier added. “I remember at the first week, Vijay took us to grab some stuff to start working.”

“All we had to do was go next door and find people,” Honore concluded.

SRL Members Embark on Summer Opportunities

This summer, the members of the Sketch Recognition Lab are going statewide and nationwide to participate in research and internship opportunities.

Many students will be remaining in the Texas A&M University lab to continue working on research, projects, and conference submissions. Others will be traveling far and wide to work.
For example, Siddhartha Karthik is interning at Yahoo Inc, in Sunnyvale, California for the summer. While he was excited about working in the beautiful Silicon Valley, Karthik was equally intrigued by the prospect of working with Yahoo! CEO Marissa Mayer, whom he had heard a great deal of praise for.

(Siddartha Karthik and Marissa Mayer)

"I am part of the Network Security team," Karthik elaborated.

While working under David Filo, the team will be "pushing code that actually impacts 500 millions users on a daily basis."

When asked how his summer opportunity relates to his work, Karthik replied, "I have always been interested in Network Security. This internship will give me hands on experience on one of the largest networks in the industry."

All the while, Ranerio Lara-Garduno will be spending a large part of his summer at the Harvard School of Public Health in Boston for his first research internship.

By combining information from GIS and GPS systems found in cellphones, Lara-Garduno will be researching technologies that can not only locate someone, but also be able to tell whether that person is inside or outside of a building – something that global position systems (GPS) cannot currently do.

Both Lara-Garduno’s research in SRL and his summer research will be dealing with the fusion of public health and computer science. While Lara-Garduno is enthusiastic about his summer opportunity, one of the things he is most excited to do is spend time in Boston. He attended the TAPIA conference in the same city, but was unable to explore due to the winter storms and isolated location of the conference.

“I like the idea of going to a new city and getting to know it for a bit,” he explained. “And, of course, the research connections I’m going to make there are going to be second to none.”

Lara-Garduno went on to explain that what drew him to this opportunity is how unique it was for him.

“I don’t think that it’s something you would normally see for research labs and summer research. It’s research in a much more applied sense – I’m not doing something in a computer science department,” Lara-Garduno explained. “My specific research is an applied research, which is different from a lot of computer science projects.”

Larry Powell, meanwhile, will be working a 12 week internship with Freescale. He will be working and researching post-silicon verification. While he was unable to speak about much of the content of his summer opportunity, he stated that he’s excited for the experience.

“The knowledge is definitely going to be useful,” Powell stated.

He is also planning to work on his main research during the summer as well and has made a deal with Freescale to be able to do his research when he is not working on his internship projects.

Across the country, Folami Alamudun is doing research work in Oak Ridge National Lab in Oak Ridge Tennessee.

 (Folami Alamudun on a hike in the Smoky Mountains)

Alamudun’s research in SRL focuses on gaze interpretation and tracking in radiologists performing mammograms. He hopes to help cut down human error in misdiagnosis. His summer opportunity perfectly synchronizes with his interests.

“I will be analyzing radiologists' gaze data collected during mammographic screening,” Alamudun explained. “This summer, I am trying to extract additional features to build a predictive model for diagnostic error.”

While the prospect of hiking in the Smoky Mountains every weekend greatly excites him, Alamudun was equally attracted to the national lab for another reason as well:

“The idea of working in a national lab with so many brilliant scientists (and history) is thrilling,” he commented. “I get to walk past Titan [a massive supercomputer] every day. That never gets old.”
Similarly, Seth Polsley will be working at the University of Kansas in the Speech and Applied Neuroscience Lab, an area of study that is different from his main projects with SRL – Mechanix and Persketchitivity.

“I’ll be working on application for children with speech apraxia and other speech disorders,” Polsley explained his summer work. “Basically it’s an application where they can drag and drop words and things and they can move how long they are and where they are placed up and down. They can practice intonation and learn how tone works.”

After working at the SAN Lab, Polsley plans to work on his thesis either locally or in Kansas.

“I really, really want to start a nonprofit this summer with my sister,” Polsley commented when asked what he was most excited for this summer. “Because my thesis ties in with her stuff as well.”

He went on to explain that he would like the nonprofit, tentatively titled “Upright”, to be an educational or assisted living organization. Upright would provide technology and frames that provides rehabilitation assistance to people who are paralyzed from serious injuries.

“There is a big gap where if you don’t show notable improvement within a month and a half or so of [physical] rehab, they have to let you go and they tell you that you’ll never get better and you’ll never be able to do anything again,” Polsley explained. 

The issue presented in medical rehabilitation that Polsley and his sister want to help with is that science has proven that is not until two years after the initial incident that caused nerve damage that the body begins to repair itself and rehabilitation works best, but after just a few months, the medical system discharges people who are not showing notable improvements.

“It works well with my thesis topic since I want to use gesture and natural sensing methods to aid in rehabilitation,” he added. “I want to do rehabilitation research and I really want to start a nonprofit. It shouldn’t be something like you get into an accident and it takes away your life. This is the 21st century. We should have the technology and be able to say ‘we can help you.’”

Where ever this summer has led members of the Sketch Recognition Lab, everyone is hard at work finishing projects, fixing programs, advancing technology, and planning lab outings to de-stress and enjoy one another's company.

TEES 100th Anniversary and Dr. Hammond's Presentation to Legislative Correspondent Michael Sobolik

The TEES conference on April 20th and 21st marked the 100th anniversary of the Texas A&M Engineering Experiment Station (TEES). Dr. Tracy Hammond, the director of the Sketch Recognition Lab at Texas A&M University, attended the reception in Washington DC that celebrated the achievements of researchers.

The anniversary consisted of two events - a reception with the theme of Homeland Security and a round-table panel and discussion titled "The Role of Cybersecurity in Protecting the Nation's Infrastructure" that was held the next day.

The discussion revolved around how researchers and research universities can train future workers to be capable of defending resources in a digital world and resolve issues faced by current agencies. 

"Cybersecurity is a challenge that affects us all," Dr. M. Katherine Banks, the director of TEES, was quoted saying, "It is critical that we raise public dialogue on this topic by bringing together our engineering experts with public policy leaders."

Moderating the panel were Dr. Valerie Taylor, the senior associate dean for academic affairs and Royce E. Wisenbaker, a professor at Texas A&M University. Along with Dr. Pierce Cantrell and Dr. Daniel Ragsdale from TAMU, the panel included Dr. John Launchbury - the program manager for Defense Advanced Research Projects Agency, and Dr. Victor Piotrowski - the program director for the National Science Foundation. US Rep. John Ratcliffe attended.

The reception was hosted by Chancellor Sharp, Dean Banks, and the Center on Competitiveness. Featured at the reception were demonstrations by Dr. John Valasek (UAS Surveillance), Dr. Robin Murphy (Infectious Diseases), Dr. Jim Wall (electronic Warfare Technology), and Dr. Tracy Hammond (Mobile Technology for Military). The research displayed attracted people from Capitol Hill, nonprofit organizations, and federal agencies. 

“For 100 years, the Texas A&M Engineering Experiment Station has performed groundbreaking engineering research and developed technology to address state and national problems,” TEES website states.

TEES partners with industry, communities, and academic institutions to improve the quality of life and promote development in the economy. They create new technology and help resolve problems in health and environment.

To further research, TEES has centers in specialized areas such as education and training, energy systems, healthcare, information systems and sensors, safety and security, and materials and manufacturing. They gain funding from federal agencies including the Department of Defense, the Department of Energy, the National Science Foundation, and NASA.

“In addition, as an institute of higher education, we work to improve student performance in math, science and technology,” says TEES website.

Their mission is “to produce and transfer the highest quality, relevant engineering and technology-oriented research by leveraging capabilities statewide in order to:

• Improve economic development and quality of life in Texas and the nation
• Enhance educational systems
• Support interdisciplinary fundamental and applied research
• Transfer technology from research and development activities to useful applications
• Commercialize promising technologies.”

While attending the 100th anniversary reception, Dr. Hammond presented highlights of her government funded research on Homeland Security to Senator Cruz’s Legislative Correspondent, Michael Sobolik.

(Dr. Hammond's poster for her presentation to Mr. Sobolik and the TEES reception)

Her presentation covered her research about GeoTrooper – the haptic navigation vest for paratroopers. Highlighted in her presentation was the incredible travel-time difference that occurred when GeoTrooper was introduced. It cut down the time for a soldier to reach their desired destination from around two hours to about 10 minutes.

With up to 2,000 soldiers jumping from planes at a time, this difference is even more important for keeping large amounts of paratroopers safe.

Furthermore, the hands-free navigation allows for soldiers to stay alert to their surroundings – which on the battlefield is absolutely necessary and important.

Dr. Hammond has also done research with GeoTrooper for more recreational reasons. Dubbed HaptiMoto, the vest can also be used to help guide motorcyclists safely to their destinations.

SRL congratulates TEES for celebrating its 100th birthday! Here’s to 100 more!