Genetic Sequence Matching Using D4M Big Data Approaches

Abstract

Recent technological advances in Next Generation Sequencing tools have led to increasing speeds of DNA sample collection, preparation, and sequencing. One instrument can produce over 600 Gb of genetic sequence data in a single run. This creates new opportunities to efficiently handle the increasing workload. We propose a new method of fast genetic sequence analysis using the Dynamic Distributed Dimensional Data Model (D4M) - an associative array environment for MATLAB developed at MIT Lincoln Laboratory. Based on mathematical and statistical properties, the method leverages big data techniques and the implementation of an Apache Acculumo database to accelerate computations one-hundred fold over other methods. Comparisons of the D4M method with the current gold-standard for sequence analysis, BLAST, show the two are comparable in the alignments they find. This paper will present an overview of the D4M genetic sequence algorithm and statistical comparisons with BLAST.

6 pages; to appear in IEEE High Performance Extreme Computing (HPEC) 2014

References in corpus (1)

• D4M 2.0 Schema: A General Purpose High Performance Schema for the Accumulo Database

Cited by in corpus (8)

• Graphs, Matrices, and the GraphBLAS: Seven Good Reasons
• GraphChallenge.org: Raising the Bar on Graph Analytic Performance
• Julia Implementation of the Dynamic Distributed Dimensional Data Model
• Design, Generation, and Validation of Extreme Scale Power-Law Graphs
• GraphChallenge.org Triangle Counting Performance
• Rapid Sequence Identification of Potential Pathogens Using Techniques from Sparse Linear Algebra
• Implementing Suffix Array Algorithm Using Apache Big Table Data Implementation
• Using a Big Data Database to Identify Pathogens in Protein Data Space