Article

BigMPI4py: Python module for parallelization of Big Data objects

2019-01-17

Abstract excerpt

Big Data analysis is a discipline with a growing number of areas where huge amounts of data is extracted and analyzed. Parallelization in Python integrates Message Passing Interface via mpi4py module. Since mpi4py does not support parallelization of objects greater than 2 31 bytes, we developed BigMPI4py, a Python module that wraps mpi4py, supporting object sizes beyond this boundary. BigMPI4py automatically dete...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: 9d85f257-36bf-50a4-90d2-8f7693715c76
DOI: 10.1101/517441

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

BigMPI4py: Python module for parallelization of Big Data objectsDOI 10.1101/517441Read

HydraMPP: A lightweight library for distributed massive parallel processing in Python - threading at scaleSemantic proximity Read

GigaSOM.jl: High-performance clustering and visualization of huge cytometry datasetsSemantic proximity Read

SLOW5: a new file format enables massive acceleration of nanopore sequencing data analysisSemantic proximity Read

SLOW5: a new file format enables massive acceleration of nanopore sequencing data analysisSemantic proximity Read

Pytometry: Flow and mass cytometry analytics in PythonSemantic proximity Read

GenMPI: Cluster Scalable Variant Calling for Short/Long Reads Sequencing DataSemantic proximity Read

DiGeST: Distributed Computing for Scalable Gene and Variant Ranking with Hadoop/SparkSemantic proximity Read

Methods for high-throughput MethylCap-Seq data analysisSemantic proximity Read

quickBAM: a parallelized BAM file access API for high throughput sequence analysis informaticsSemantic proximity Read

Spark4VCF: A Novel Big Data Framework to Accelerate Genomics AnalysisSemantic proximity Read

MaRe: Processing Big Data with application containers on Apache Spark.Semantic proximity Read

Database Resources of the BIG Data Center in 2018.Semantic proximity Read