Modern file storage accelerates the AI-driven search for cures

Modern file storage powers the life sciences

Both genomic sequencing and Cryo-EM are data-centric and data-rich endeavors. They both utilize AI with ML workflows that include data collection, preparation, model building, training, and inferencing. The massive amounts of data involved present huge challenges to compute and data storage capabilities in terms of scalable performance and capacity, ease of management, headroom for growth, and support.

To optimize these workloads, you need computational and data storage resources that can crunch through massive amounts of data. Research organizations must invest in high-performance, GPU-powered servers. They also need modern file storage capable of feeding data to the GPUs fast enough to keep up. So the performance of the file system matters a great deal. In practice, you need a file system with performance scalable to exabyte capacities to keep your GPU-driven servers busy.

Legacy file systems will not be able to meet those demands, as they come with unavoidable trade-offs in performance, scalability, simplicity, feature set, and cost. The other alternative, public cloud storage, has drawbacks that include security, performance and latency, cost at scale, data sovereignty and management, flexibility, and egress fees. These limitations have pushed organizations to look for on-premises file storage that eliminates trade-offs and also delivers an intuitive cloud experience. Let’s detail what we really need in file storage for life sciences and healthcare.

Performance at scale that’s affordable: Your file storage solution must deliver performance at scale — for all your data, not just some of it — and it must be fast despite working with huge amounts of data. Moreover, it must be affordable, even while delivering performance at scale for both sequential and random I/O operations, the latter being common in genomic analysis. Your file storage cannot achieve affordability via tiering because spinning disks will introduce rotational delays. Tiers create additional problems, as you will need to move data around between faster and slower storage and constantly tune performance. Even with all that overhead, legacy file storage can’t guarantee that you will be able to access and store the right data in the right place, at the right time, and with the right performance.

Simple to use with enterprise features: Researchers and scientists need file storage that can be set up and run easily, so that they can focus on strategic initiatives rather than being weighed down by day-to-day operations. Life sciences researchers have enough complexity to deal with in their field. They should not be burdened with the additional complications of managing traditional file storage infrastructure. And they want file storage that is not only fast but has enterprise features such as native replication, no-overhead snapshots, and a multiprotocol, global namespace to facilitate collaboration.