Please use this identifier to cite or link to this item:
Title: A data reduction and compression description for high throughput time-resolved electron microscopy
Authors: Datta, Abhik 
Ng, Kian Fong 
Balakrishnan, Deepan 
Ding, Melissa
Chee, See Wee 
Ban, Yvonne
Shi, Jian 
Loh, N. Duane 
Issue Date: 28-Jan-2021
Publisher: Nature Research
Citation: Datta, Abhik, Ng, Kian Fong, Balakrishnan, Deepan, Ding, Melissa, Chee, See Wee, Ban, Yvonne, Shi, Jian, Loh, N. Duane (2021-01-28). A data reduction and compression description for high throughput time-resolved electron microscopy. Nature Communications 12 (1) : 664. ScholarBank@NUS Repository.
Rights: Attribution 4.0 International
Abstract: Fast, direct electron detectors have significantly improved the spatio-temporal resolution of electron microscopy movies. Preserving both spatial and temporal resolution in extended observations, however, requires storing prohibitively large amounts of data. Here, we describe an efficient and flexible data reduction and compression scheme (ReCoDe) that retains both spatial and temporal resolution by preserving individual electron events. Running ReCoDe on a workstation we demonstrate on-the-fly reduction and compression of raw data streaming off a detector at 3 GB/s, for hours of uninterrupted data collection. The output was 100-fold smaller than the raw data and saved directly onto network-attached storage drives over a 10 GbE connection. We discuss calibration techniques that support electron detection and counting (e.g., estimate electron backscattering rates, false positive rates, and data compressibility), and novel data analysis methods enabled by ReCoDe (e.g., recalibration of data post acquisition, and accurate estimation of coincidence loss). © 2021, The Author(s).
Source Title: Nature Communications
ISSN: 2041-1723
DOI: 10.1038/s41467-020-20694-z
Rights: Attribution 4.0 International
Appears in Collections:Elements
Staff Publications

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
10_1038_s41467-020-20694-z.pdf3.1 MBAdobe PDF




checked on Nov 29, 2022

Page view(s)

checked on Dec 1, 2022

Google ScholarTM



This item is licensed under a Creative Commons License Creative Commons