Vision-Based Automation for Accelerated Structural Interpretation in Atomic-Resolution Microscopy

Schwenker, Eric

doi:https://doi.org/10.21985/n2-7rvp-gh89

Work

Vision-Based Automation for Accelerated Structural Interpretation in Atomic-Resolution Microscopy

Public

Download PDF

Download All Files (.zip)

At its core, the purpose of microscopy is to make objects and their underlying structures visible under high magnification. With the remarkable progress of electron microscopy, the sub-micron “high” magnification of light microscopy has been completely refashioned to encompass subatomic length scales. Unfortunately, higher-magnification does little to negate existing interpretability challenges present in images of crystal imperfections– which comprise some of the most scientifically intriguing and technologically relevant materials images. And any sort of direct interpretation advantage that this localized imaging affords, is quickly overwhelmed by the sheer volume of images that must be processed to extrapolate effects to the bulk of the crystal. Fortunately, computer vision has emerged as a tool for automated analysis and interpretation of images from large volumes of complex and/or noisy visual inputs – conditions nearly synonymous with images collected at atomic resolution. This thesis is focused on the development of vision-based automation pipelines with applications specific to structural interpretation of electron microscopy images. Moreover, it is shown that vision-based automation can be used to harmonize the power and scaling of computation, with the quantitative insights now accessible via experimental imaging, physics-based modeling and simulation, and even peer-reviewed scientific literature. We begin by exploring methods for quantifying image similarity in atomic-resolution microscopy. Image similarity is an essential consideration in the general interpretation efforts, as the intensity signals from the experimental micrograph often must be compared to simulation to better understand acquisition parameters or validate proposed structures. Second, we focus on the problem of determining 3D atomic structure from experimental STEM and STM images and develop custom automation tools to find candidate structures that are both energetically feasible and produce images consistent with what is observed experimentally. Finally, we highlight the development of a pipeline for constructing self-annotated microscopy datasets from scientific literature. It is our vision that the pipelines developed here will help enable meaningful automation in the structural interpretation of atomic-resolution microscopy images, both as a mechanism for suggesting plausible structures that match experimental observations, and as a first step in translating recorded scientific knowledge from existing images to future images unseen.

Creator