Invited Speakers

Dr. Henry Ruhl, National Oceanography Centre, Southampton, UK

Abstract

Advances in Computer Vision and Pattern Recognition for Research in Biological Oceanography

Henry A Ruhl, Jennifer M Durden, Daniel OB Jones, Timm Schoening, Tim W. Nattkemper, Danelle Cline and Duane Edgington

A convergence of advancements in marine autonomous systems platforms, optical sensors and cameras, as well as computer pattern recognition is facilitating a step change in the quantity and quality of data for understanding ocean biogeochemistry and ecosystems. Ocean biology plays a critical role in regulating the amount of carbon dioxide (CO₂) in the Earth’s atmosphere. The so-called ‘biological carbon pump’ operates by capturing carbon dioxide during photosynthesis and primary production of biomass in the upper most part of the ocean, where sunlight penetrates. A portion of this newly produced organic carbon biomass forms ‘marine snow’ that sinks to the deep sea. Importantly, atmospheric CO₂ levels are sensitive to how sinking marine snow particles are broken down as they sink. Moreover, marine snow also forms the food that fuels most deep sea organisms. Here we present case studies on the biological oceanographic research incorporating computer vision as well as a discussion of promising approaches that are in development for the following scientific themes: 1) understanding the uptake of carbon into the upper ocean, 2) the modification and depth of sequestration of organic carbon in the water column, and 3) organic carbon’s influence on seafloor biology in space and time. A detailed understanding of the biological carbon pump is currently limited by a lack of observations that span the necessary time and depth ranges to measure the process. For example, most high-quality observations and sampling of the biological carbon pump mechanisms come from research expeditions that typically last from a week to a month. Computer vision and annotation of image data collected by autonomous systems will mean that such observations can be made over months to years. The resulting oceanographic understanding will be of interest to policy groups such as the United Nations (UN) – Intergovernmental Panel on Climate Change (IPCC) and the UN - International Seabed Authority (ISA).

Bio

Dr. Henry Ruhl is Associate Head, Ocean Biogeochemistry and Ecosystems Group at the National Oceanography Centre of the United Kingdom and an Adjunct Researcher at the Monterey Bay Aquarium Research Institute. His scientific contributions have outlined links between changes in climate conditions and subsequent changes in surface productivity, organic carbon fluxes into the deep sea, and the response of abyssal fauna to those fluxes in terms of distribution, abundance, and community structure in photographic data and sediment samples. He has led the Autonomous Ecological Surveying of the Abyss (AESA) project, which has been featured by the BBC and New Scientist. A key focus of his current research is taking advantage of new and developing optical imaging technologies to tackle long-standing challenges in assessing the role of biological systems in controlling the stocks and flows of carbon in the ocean. He is also a member of several ocean observatory steering and advisory committees including the European Multidisciplinary Seafloor and water column Observatory (EMSO), the Fixed-point Open Ocean Observatories (FixO3) project and Ocean Networks Canada.

Dr. Anthony Hoogs, Kitware, Clifton Park, NY, USA

Abstract

Video and Imagery Analytics for the Marine Environment (VIAME): an Open Source Framework for Underwater Image Processing

Keith Fieldhouse, Anthony Hoogs, Linus Sherrill, Matt Dawkins

The Video and Image Analytics for the Marine Environment (VIAME) framework is being developed by Kitware to be an open source, comprehensive system for underwater video and imagery analysis that enables rapid, low-cost integration of new algorithmic modules, new datasets and new end-user capabilities. Funded by NOAA and overseen by the NOAA Automated Imagery Analysis Strategic Initiative committee, VIAME allows researchers to develop marine-oriented video analytics within a common framework, leveraging a common set of tools, techniques and utilities. VIAME's dynamic, plugin oriented pipeline framework will allow analytics and other tools to be combined freely for experimentation and evaluation. It includes evaluation tools such as metrics scoring software that implements a variety of common detection and classification measures. For data analysis and exploitation, VIAME includes substantial visualization capabilities to display, explore and understand information derived from images and video in various contexts. More information is available at KWIVER.org.

Bio:

Dr. Anthony Hoogs is the Senior Director of Computer Vision at Kitware, a small software R&D firm based on open source software for scientific applications. Dr. Hoogs leads Kitware's Computer Vision group, which he started in 2007 and now has 25 members including 12 PhDs. He has led more than two dozen government-sponsored contracts in video and motion analysis, involving more than 20 universities and multiple government institutions.

Since 2014 he has served on the NOAA Automated Imagery Analysis Strategic Initiative Steering Committee, after being an organizer of the National Research Council Workshop on Robust Methods for the Analysis of Images and Videos for Fisheries Stock Assessment, sponsored by NOAA. Previously at GE Global Research (1998- 2007), Dr. Hoogs led a team of researchers in video and imagery analysis on projects sponsored by the US Government and GE. For more than two decades, he has supervised and performed research in various areas of computer vision including event, activity and behavior recognition; motion analysis and tracking; object detection; and content-based retrieval.

He has published more than 70 papers in computer vision, and has served as the Co-Chair (with Ben Richards) for the Workshop on Automated Analysis of Video Data for Wildlife Surveillance 2015 and 2016; General Co-Chair for CVPR 2017; Computer Vision Foundation Advisory Board; Workshops Co-Chair, Corporate Relations Chair and Area Chair for CVPR; General Co-Chair for the Winter conference on Applications of Computer Vision (WACV) 2016; program Co-Chair for WACV 2009 and 2011; and is on the steering committee for WACV.

Dr. Hoogs received a Ph.D. in Computer and Information Science from the University of Pennsylvania in 1998; an M.S. from the University of Illinois at Urbana-Champaign in 1991; and a B.A. from Amherst College in 1989.

Dr. Yogesh (Yogi) Girdhar, Woods Hole Oceanographic Institution (WHOI), Woods Hole, MA, USA

Abstract

Realtime unsupervised scene understanding for building curious underwater exploration robots

This talk presents a novel approach to modeling visual curiosity in a mobile robot, which is useful for monitoring and adaptive data collection tasks, especially in the context of long term autonomous missions. We propose a realtime inference algorithm to learn a semantic representation of the environment, using which, we plan a path through the locations in the world with high semantic information content. The life-long learning behavior of the proposed perception model makes it suitable for long-term exploration missions. We validate the approach using Aqua robot, deployed in a coral reef, and using simulated exploration experiments using real data. We find that the proposed exploration paths that are biased towards locations with high topic perplexity produce better terrain models with high discriminative power. Moreover, we show that the proposed algorithm implemented on Aqua robot is able to do tasks such as coral reef inspection, diver following, and sea floor exploration, without any prior training or preparation.

Bio

Dr. Yogesh (Yogi) Girdhar is an Assistant Scientist at the Applied Ocean Physics & Engineering department at Woods Hole Oceanographic Institution (WHOI). He received his BS and MS from Rensselaer Polytechnic Institute, where he was awarded Paul A. McGloin prize for most outstanding academic achievement in Computer Science. He received his PhD from McGill University in 2014, with thesis title “Unsupervised Semantic Perception, Summarization, and Autonomous Exploration for Robots in Unstructured Environments”, which received the Honorable Mention for CIPPR Doctoral Dissertation Award. After his PhD, Yogi was a Devonshire Postdoctoral Scholar, and a FQRTN Postdoctoral Fellow at WHOI.

Marine robots live in a communication starved environment where autonomy is an absolute requirement for their deployment. Yogi’s ongoing research focuses on the algorithmic challenges that lie on the intersection of field robotics and machine learning. More specifically he is interested in the use of unsupervised machine learning algorithms for rapid analysis of large datasets, and for building autonomous exploration robots that can prioritize their sampling based on high level environmental context.