Decentralized storage in brief

Web3.Storage relies on decentralized storage to store your files for free on the network. Decentralized storage is a technique for storing data that, instead of traditional servers, uses a distributed network with many participants providing storage capacity. This model inherently builds in redundancy that provides resilience against failure and attacks, as well as enhanced performance due to the geographical localization offered by a large network of distributed storage providers. While you don't need to understand decentralized storage to be able to incorporate Web3.Storage in your apps and services, if you're curious about what's going on under the hood, read on.

Content persistence

Broadly speaking, much of today's web operates using what is known as location addressing. Location addressing retrieves online information from specific locations on the web ─ i.e. from behind URLs.

https://example.com/page-one.html

However, this approach has some critical problems. Location addressing is centralized, meaning that whoever controls that location controls the content. The controller can change the content, completely replace it, or just take it away. This means location-based addresses are vulnerable to attacks, exploitation, and loss.

One part of an approach to decentralizing this legacy way of operating the web entails implementing a new kind of addressing: content addressing. Content addressing is the technique of issuing each piece of data a content identifier (CID), which is a token derived directly from the file's contents using an algorithm that will always generate the same key for the same content. Using content addressing, files can be queried and retrieved based on what they are, not where they are — one major ingredient in enabling the web to break free from centralized content control. However, content addressing is only part of the solution.

Web3.Storage uses CIDs generated by IPFS, the InterPlanetary File System, to enable content addressing for all data stored in its network — however, just because a file has a CID doesn't mean that the file is guaranteed to be around forever. In a well-operating decentralized system, participants all need to agree to be good actors and provide reliable storage capacity. For this, Web3.Storage uses the Filecoin network. The Filecoin network and its namesake token FIL (or ⨎ for short) were created to incentivize storage providers on the network to agree to storage deals. These deals specify that a certain amount of storage capacity will be provided over an agreed period of time, ensuring the second part of the solution: content persistence.

Filecoin for verifiable content persistence

Leveraging the Filecoin network for data stored using Web3.Storage ensures that content is available for retrieval, thus assuring that the content-based addressing provided by IPFS remains resilient over time. Filecoin achieves this mission using a number of methods, including novel cryptography, consensus protocols, and game-theoretic incentives — but perhaps the most important of these is Filecoin's unique approach to storage verification.

Filecoin's storage verification system solves a previously intractable problem for decentralized storage: How can storage providers prove that they are really storing the data they say they are over time? Filecoin's proving algorithms take care of this verification:

  • Proof-of-Replication proves that a given storage provider is storing a unique copy of a client's original data.
  • Proof-of-Spacetime proves that the client's data is stored continuously over time.

In addition to this proof system, the Filecoin network also relies on game-theoretic incentives to discourage malicious or negligent activity. In order to become a Filecoin storage provider, all potential providers must provide collateral in the form of FIL when agreeing to a storage deal. Additionally, any storage provider that fails Proof-of-Spacetime checks is penalized, loses a portion of their collateral, and is eventually prevented from offering storage to clients again.

Storing and monitoring with Web3.Storage

How can you see these principles in action with your files uploaded to Web3.Storage? It's easy using the Web3.Storage JavaScript client library.

When you upload a file to Web3.Storage using the put() method, you get the CID of that file in return. The file is then put into a queue for geographically distributed Filecoin network storage providers who have been chosen for performance and availability. These providers bid on the right to store the files in the queue — including your file — and agree to a storage deal.

You can monitor this activity for files you upload to Web3.Storage by calling status() and providing a file's CID. This will return a list of deals that have been made at the time of query. Here's how you might include this call in your JavaScript project:

// get files from form object
const fileInput = document.querySelector('input[type="file"]');

// store files, obtain CID
const rootCid = await client.put(fileInput.files, {
  name: 'my files',
  maxRetries: 3,
});

// query status based on CID
const info = await client.status(rootCid);

// display results of query
console.log(`${info.cid} ${info.dagSize} ${info.created}`);
for (const deal of info.deals) {
  console.log(`${deal.id} -- ${deal.status}`);
}

Summary

The legacy approach of using location addressing to reference files on the web has a number of critical vulnerabilities to interference, exploitation, and loss. Because whoever controls the location controls the content, these location-addressed, centrally stored networks also leave the web exposed to the whims of big, corporate central storage platforms.

By pairing IPFS content addressing (to decentralize how resources are specified) with Filecoin (to decentralize how storage capacity is obtained), it becomes possible to create a complete solution for locating, storing, and obtaining data — one that is not only resilient against these vulnerabilities, but also rewarding for network participants.

Web3.Storage combines the content addressing capabilities of IPFS and the proving algorithms and storage deal marketplace of Filecoin in order to achieve this complete solution, wrapping the power of these two systems into an easy-to-use service with simple, familiar methods such as put(), get(), and status(). Even better, Web3.Storage is free to use and integrate into your apps and services.

Learn more

Want a deep dive into decentralized storage, how it works, and why it's important? Check out ProtoSchool for an in-depth look at verifying storage on Filecoin, plus a wealth of other interactive tutorials on DWeb concepts, protocols, and tools.

Was this information helpful?

Help us improve this site!