When Is Decentralized Storage the Right Choice?
The amount of data created has doubled every year, presenting a host of challenges for organizations: security and privacy issues for starters, but also storage costs. What situations call for that data move to decentralized cloud storage rather than on-prem or even a single public cloud storage setup? What are the advantages and challenges of a decentralized cloud storage solution for data, and how can those be navigated?
On this episode of The New Stack Makers podcast, Ben Golub, CEO of Storj, and Krista Spriggs, software engineering manager at the company, were joined by Alex Williams, founder and publisher of The New Stack, along with Heather Joslyn, TNS’ features editor. Golub and Spriggs talked about how decentralized storage for data makes sense for organizations concerned about cloud costs, security, and resiliency.
When Is Decentralized Storage the Right Choice?
The best situations for using Storj or other companies that offer decentralized data storage, Golub said, are those in which cloud costs need to be minimized, where security is a special concern, and where keeping the data close to where it’s being created and consumed matters most.
“What we’re finding is for something like 80% of the data that’s being created, and where most of the growth is, this decentralized approach just makes sense,” the CEO said.
Security concerns alone make a strong case for decentralized storage, Golub said. “Anytime you’re storing data in clear text in a centralized location, it’s one mistake or one bad hacker away from being compromised,” he said.“And ultimately I think we probably don’t want a world where 80% of the cloud is controlled by three of the largest companies on the planet who happen to all be in the business of selling data.”
At Storj, the DevOps team Spriggs oversees “drink their own champagne,” as she said, keeping the most recent 12 hours data in “hot files” and sending the rest to Storj DCS. It’s important for organizations to keep data long term, she noted, because its usefulness isn’t always apparent when it’s first created and collected.
Take, for instance, data lakes, Spriggs said: “A lot of that data, you don’t know how valuable it is until you have a question. And if you don’t store all of the data that you have available to store, you might be painting yourself into a corner where you can’t ask the type of question that you want to ask. And you’ll never get that data back.”
By contrast, being able to easily locate old data, she said, “really unlocks different types of problems that you can solve, because you suddenly don’t have this big blocker to face.”
Decentralized data storage, Golub said, is destined to become more widely adopted as organizations themselves grow more distributed. “If we look back in 10 years we’ll see that decentralized cloud in general, not just decentralized storage, makes sense — the same kind of sense that we realized that decentralized telecommunications did 20 years ago, when the internet came about right. It is inherently faster, inherently better, inherently more scalable and inherently more flexible.”
