CSEE faculty receive NSF award to help robots learn tasks by interacting naturally with people

UMBC Assistant Professors Cynthia Matuszek (PI) and Francis Ferraro (Co-PI), along with senior staff scientist at JHU-APL John Winder (Co-PI) received a three-year NSF award as part of the National Robotics Initiative on Ubiquitous Collaborative Robots. The award for Semi-Supervised Deep Learning for Domain Adaptation in Robotic Language Acquisition will advance the ability of robots to learn from interactions with people using spoken language and gestures in a variety of situations.

This project will enable robots to learn to perform tasks with human teammates from language and other modalities, and then transfer what they have learned to other robots with different capabilities in order to perform different tasks. This will ultimately allow human-robot teaming in domains where people use varied language and instructions to complete complex tasks. As robots become more capable and ubiquitous, they are increasingly moving into complex, human-centric environments such as workplaces and homes.

Being able to deploy useful robots in settings where human specialists are stretched thin, such as assistive technology, elder care, and education, has the potential to have far-reaching impacts on human quality of life. Achieving this will require the development of robots that learn, from natural interaction, about an end user’s goals and environment.

This work is intended to make robots more accessible and usable for non-specialists. In order to verify success and involve the broader community, tasks will be drawn from and tested in community Makerspaces, which are strongly linked with both education and community involvement. It will address how collaborative learning and successful performance during human-robot interactions can be accomplished by learning from and acting on grounded language. To accomplish this, the project will revolve around learning structured representations of abstract knowledge with goal-directed task completion, grounded in a physical context.

There are three high-level research thrusts: leverage grounded language learning from many sources, capture and represent the expectations implied by language, and use deep hierarchical reinforcement learning to transfer learned knowledge to related tasks and skills. In the first, new perceptual models to learn an alignment among a robot’s multiple, heterogeneous sensor and data streams will be developed. In the second, synchronous grounded language models will be developed to better capture both general linguistic and implicit contextual expectations that are needed for completing shared tasks. In the third, a deep reinforcement learning framework will be developed that can leverage the advances achieved by the first two thrusts, allowing the development of techniques for learning conceptual knowledge. Taken together, these advances will allow an agent to achieve domain adaptation, improve its behaviors in new environments, and transfer conceptual knowledge among robotic agents.

The research award will support both faculty and students working in the Interactive Robotics and Language lab on this task. It includes an education and outreach plan designed to increase participation by and retention of women and underrepresented minorities (URM) in robotics and computing, engaging with UMBC’s large URM population and world-class programs in this area.

Prof. Tülay Adali receives prestigious Humboldt Research Award for advanced data analysis

UMBC’s Tülay Adali, professor of computer science and electrical engineering (CSEE) and distinguished university professor, has received the prestigious Humboldt Research Award. The Alexander von Humboldt Foundation describes the award as presented to scholars “whose fundamental discoveries, new theories, or insights have had a significant impact on their own discipline and who are expected to continue producing cutting-edge achievements in the future.”

Adali is the director of UMBC’s Machine Learning for Signal Processing Lab. Her research focuses on developing flexible methods for data fusion. These innovative methods enable researchers to extract powerful features from multi-modal data by letting them fully interact with and inform each other.

A main application area of her work has been medical image analysis, where these features are used in diagnosis as well as treatment planning and evaluation. Adali and her research collaborators are also exploring applications of these methods in remote sensing, misinformation detection, and gesture and video analysis. 

A years-long research collaboration

Humboldt Award recipients spend up to one year conducting collaborative research at institutions in Germany. Adali plans to continue to work with her longtime collaborator Peter Schreier, who is based in Paderborn University. Through a research connection that has spanned many years, Adali says that her lab and Schreier’s continue to have wonderful synergy. 

Together, Adali and Schreier have worked to address problems such as data-driven discovery of relationships in multi-modal data, and in particular, when the sample sizes are small. “This is a key practical problem in many applications, especially in the medical domain,” Adali shares. She notes that this provides an important starting point for their current work. 

“Things are moving along, even though I could not travel this summer, as we started having weekly research meetings between our groups,” Adali says. “This is a valuable experience for my students. In the past, we had hosted Schreier and his students here at UMBC, some of my students had met Schreier and his students at conferences before, and these initial physical connections matter. I am hoping we will all be able to travel again, soon.”

Receiving the award

As a Humboldt Award recipient, Adali was invited to attend a gathering in June with her fellow awardees, hailing from universities around the world. Due to COVID-19, the event was moved online. Awardees had an opportunity to meet the German president virtually as part of the event. 

While she wishes the event could have been held in person, Adali says that it gave her an exciting opportunity to connect with other Humboldt awardees and learn more about scientist and explorer Alexander von Humboldt.

In 2015, Curtis Menyuk, professor of CSEE, received a Humboldt Award.

Adapted from a UMBC News article written by Megan Hanks. Banner image: Tülay Adali, fourth from left, with the members of her lab. Photo courtesy of Adali.

Two UMBC student teams win USM COVID-19 app challenge

Earlier this summer, the University System of Maryland (USM) COVID-19 Task Force invited members of the USM community to develop mobile apps that would help Maryland residents respond to the COVID-19 pandemic. Among the six winning teams just announced are two groups from UMBC. One team developed an app to support the healthcare of people with COVID-19. The other focused on connecting residents with dining options and restaurant policies as they change during the pandemic.

Community participation

Each of the six winning teams received a $3,000 award, provided by UMBC’s Alex. Brown Center for Entrepreneurship. The apps submitted were reviewed by a panel of judges from large corporations, start-up companies, and academia. 

Undergraduate and graduate students were invited to participate, as well as university staff, faculty, postdoctoral researchers, members of USM-affiliated startup companies, and small businesses. Winners hailed from UMBC, University of Baltimore, Towson University, and University of Maryland, College Park.

Tracking health conditions of COVID-19 patients

In the community category, Kirubel Tolosa M.S. ‘23, information systems; Pradeep Margasahayam Prakash M.S. ‘21, information systems; and Raghav Deivachilai M.S. ‘23, computer science, created an app called Follow-up. The app enables healthcare providers to track the condition of people with COVID-19 who are isolating at home. By receiving regular symptom updates, physicians and nurses are able to more easily follow-up with their patients as needed.

The Follow-up team entered the app challenge with the goal of developing an app that would help address the spread of the virus and its impact on affected individuals. At the same time, they knew they had to design and prototype their app in a short time frame, so their scope and requirements had to be manageable.

“This challenge has taught us the value of teamwork and collaboration,” said Tolosa, on behalf of the group. “We are looking forward to working on this app further to put it to use in a real-world setting.”

Supporting restaurants during COVID-19

The app Snuggrub, developed by Emily Sullivan ‘21, computer science, and Dominic Crofoot ‘19, computer science, was a winner in the student category. Sullivan and Crofoot focused on the way that many formerly full-service restaurants shifted to pick-up only service or outdoor dining during the pandemic. At the same time, dining regulations, guidance, and options began changing frequently. They developed a way for users to stay up-to-date on information about nearby restaurants without needing to contact individual businesses to ask the same questions repeatedly. 

The app allows users to stay informed and receive real-time updates while making decisions based on current information. It also supports restaurants in connecting with customers and providing them with the information they need to dine safely.

The opportunity to develop an app to help address a challenge facing people across the state was appealing to Sullivan and Crofoot because it allowed them to put their skills to the test. They met when they were both interns at the Anne Arundel County Office of Information Technology. While Sullivan is still a UMBC student (and interning with the federal government), Crofoot is currently a full stack developer for Anne Arundel County.

“Dominic and I both have experience creating applications from our jobs, but this process was totally different since we were creating something from the ground up and we were doing it with such a small team and short deadline as well,” says Sullivan. “This definitely was a learning experience in personal discipline and timeline management.”

Adapted from a UMBC News article by Megan Hanks

UMBC collaborates with MxD on cybersecurity curriculum for workers in manufacturing

UMBC researchers will collaborate with the Chicago-based MxD to develop a curriculum and online platform for manufacturing professionals to increase their cybersecurity skills and to protect manufacturing plants from cyber breaches. The work is funded by a $650,000 grant from the Office of Economic Adjustment, under the U.S. Department of Defense. 

MxD is one of 14 federally-supported institutes known collectively as Manufacturing USA. It has awarded millions of dollars to research and development projects across 35 states to advance U.S. manufacturing practices and increase global competitiveness. This UMBC collaboration will be the first initiative focused on increasing manufacturing workers’ knowledge of cybersecurity.

The content of this program is completely new, as there are no existing platforms that focus on the intersection of cybersecurity and manufacturing, says Nilanjan Banerjee, principal investigator on the grant. 

Banerjee, professor of computer science and electrical engineering (CSEE) at UMBC, shares, “The program will accelerate training of practitioners in the manufacturing industry in cybersecurity. It will also expand UMBC’s impact on cybersecurity education in the manufacturing sector.”

Intersection of cyber and manufacturing

Banerjee will collaborate with a number of colleagues at UMBC to develop a curriculum tailored for people who already work in the manufacturing industry. Project co-PIs include Donna Ruginski, executive director of cybersecurity initiatives at UMBC, and Keith J Bowman, dean of UMBC’s College of Engineering and Information Technology. Alan Sherman, professor of CSEE; Linda Olivia, assistant professor of education; and Megean Garvin, director of research and assessment for the Maryland Center for Computing Education, will assess the curriculum developed to ensure it meets program goals.

Bowman helped establish the connections between UMBC and MxD, and is eager to watch the work develop. “This project fully leverages MxD, UMBC Training Centers, and UMBC assets in cybersecurity, manufacturing, and training,” says Bowman. “I have known MxD team members, including Federico Sciammarella, president and chief technology officer of MxD, ever since its origins, and I look forward to building on this collaboration.”

The first step of the multi-phased project will identify the skills most needed to protect manufacturing facilities from cyberattacks on their computer systems and machinery. UMBC and MxD will create a short-term training program for manufacturing professionals to develop these skills. 

“People will come out of this program with a certification that shows they have the tools to be successful in a cybersecurity role in manufacturing,” said Lizabeth Stuck, senior director of MxD Learn, the institute’s workforce development arm. “This has the dual benefit of upskilling workers who may be sidelined during the COVID-19 crisis and increasing the security of U.S. manufacturers from cyber-attacks.”

Addressing current needs

Banerjee explains that the recent COVID-19 pandemic has led to increased unemployment and a need for more opportunities for workers to quickly expand their skill sets. With this in mind, the program will be designed for workers to complete in less than a year and through a web-based format.

For maximum flexibility, the platform will offer both synchronous and asynchronous material. It will be launched and led by UMBC Training Centers, a not-for-profit owned by UMBC that offers professional and technical training in areas such as cybersecurity, project management, and leadership and innovation. The platform will likely launch in late January 2021. 

“This program will have a direct impact on the Defense Industrial Base Supply Chain,” says Ruginski. “It will create a robust workforce that has the cybersecurity skills required to assist companies in staying secure in the fast-paced cybersecurity manufacturing industry.”

---

Adapted from a UMBC News article written by Megan Hanks

UMBC researchers receive NSF RAPID grant to speed COVID-19 detection through a deep neural network

A research team from UMBC and the University of Maryland School of Medicine (UMSOM) has received a Rapid Response Research (RAPID) grant from the National Science Foundation to detect COVID-19 infections earlier through computing. Aryya Gangopadhyay, professor of information systems at UMBC, is PI on the grant. He explains that this work will use machine learning to improve the speed and accuracy of COVID-19 diagnosis, helping to limit the spread of the disease.

The project will make two major contributions: it will generate high-quality Convolutional Neural Networks with 2D and 3D kernels for early detection of COVID-19 infection, and it will synthesize realistic Computed Tomography images using Generative Adversarial Networks that will be publicly available for research and practice.

Developing highly accurate screening tool and synthetic datasets

Through the year-long grant totaling approximately $150,000, researchers will design, build, and train deep neural networks to detect cases of COVID-19. Gangopadhyay says this approach has a proven track record. Deep neural networks have already been used effectively in diagnosing pneumonia. 

This research will combine the power of AI and medical imaging to solve a critical problem in infectious diseases with pandemic potential, including COVID-19 and others, explains Gangopadhyay. “Our focus for this research is COVID-19. The research is an example of multidisciplinary data science that combines expertise in different fields, such as medicine and computational research,” he says. 

Research team

Gangopadhyay notes that the research will benefit from the infrastructure, research strength, and industrial partnerships of UMBC’s Center for Accelerated Real Time Analytics (CARTA).

UMBC’s team includes Yelena Yesha, distinguished professor of computer science and electrical engineering (CSEE) and director of CARTA; Yaacov Yesha, professor of CSEE; Phuong Nguyen, research assistant professor of CSEE; David Chapman, assistant professor of CSEE; and computer science Ph.D. students Sumeet Menon and Jayalakshmi Mangalagiri. Eliot Siegel, professor and vice-chair of radiology at UMSOM and chief of imaging service at the VA Maryland Healthcare System, will contribute to the research.

The team plans to work quickly and hopes to have some results available by August. Then, the researchers will work with clinicians to validate their models and data to ensure that the tools are highly accurate in predicting COVID-19.

“We are very committed to this work,” Gangopadhyay says, recognizing the incredible potential of the research. 

Collaboration during public health crisis 

This is UMBC’s second NSF RAPID Grant responding to COVID-19. In early March, UMBC’s Charissa Cheah, professor of psychology, and Shimei Pan, associate professor of information systems, and Cixin Wang, assistant professor of school psychology at the University of Maryland, College Park, received a grant to examine the intensified discrimination experienced by Chinese-Americans in the time of COVID-19.

Cheah shared, “Knowledge from this RAPID grant will help educators, health care providers, and policymakers to proactively support targeted marginalized groups and the larger public during future emergency events.” 

Both UMBC awards demonstrate the necessity to move quickly and to collaborate strategically on research related to this public health crisis.

Adapted from a UMBC News article written by Megan Hanks.