Journal Article

PUBDB-2025-00076

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Robust and automatic beamstop shadow outlier rejection: combining crystallographic statistics with modern clustering under a semi-supervised learning strategy

Gao, Y.Extern*EMBL* ; Ginn, H.CFEL*XFEL.EU*DESY* ; Thorn, A. (Corresponding author)Extern*

2024
Wiley Bognor Regis

This record in other databases:        

Please use a persistent id in citations: doi:10.1107/S2059798324008519  doi:10.3204/PUBDB-2025-00076

Abstract: During the automatic processing of crystallographic diffraction experiments, beamstop shadows are often unaccounted for or only partially masked. As a result of this, outlier reflection intensities are integrated, which is a known issue. Traditional statistical diagnostics have only limited effectiveness in identifying these outliers, here termed Not-Excluded-unMasked-Outliers (NEMOs). The diagnostic tool AUSPEX allows visual inspection of NEMOs, where they form a typical pattern: clusters at the low-resolution end of the AUSPEX plots of intensities or amplitudes versus resolution. To automate NEMO detection, a new algorithm was developed by combining data statistics with a density-based clustering method. This approach demonstrates a promising performance in detecting NEMOs in merged data sets without disrupting existing data-reduction pipelines. Re-refinement results indicate that excluding the identified NEMOs can effectively enhance the quality of subsequent structure-determination steps. This method offers a prospective automated means to assess the efficacy of a beamstop mask, as well as highlighting the potential of modern pattern-recognition techniques for automating outlier exclusion during data processing, facilitating future adaptation to evolving experimental strategies.

---

Contributing Institute(s):

1. CFEL-Coherent X-Ray Imaging (FS-CFEL-1)
2. Fachgruppe DNMX (FS-CFEL-1-DNMX)

Research Program(s):

1. 633 - Life Sciences – Building Blocks of Life: Structure and Function (POF4-633) (POF4-633)
2. VH-NG-19-02 - Working with RoPE: Representation of Protein Entities (2023_IVF-VH-NG-19-02) (2023_IVF-VH-NG-19-02)
3. DFG project G:(GEPRIS)390715994 - EXC 2056: CUI: Advanced Imaging of Matter (390715994) (390715994)

Experiment(s):

1. No specific instrument

Appears in the scientific report 2024

---

Database coverage:
; ; ; BIOSIS Previews ; Biological Abstracts ; Clarivate Analytics Master Journal List ; Current Contents - Life Sciences ; DEAL Wiley ; Ebsco Academic Search ; Essential Science Indicators ; IF < 5 ; JCR ; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection

---