limitless storage
storage space is not something people usually think about except when its nearly depleted. nowadays most laptops come with 128gb, 256gb, 512gb or even 1tb of ssd storage, with the lower options sufficient for most people, but for others (like myself) those lower options are not enough, and the higher capacity drives are just too expensive. the m1 macbook air, as of July 2022, would cost, from its base price of 999£, an additional 200£ to move from the 256gb ssd to the 512GB ssd, and an additional 400£ to move to the 1tb model, so, because i cannot afford these options i am constantly left needing more storage capacity. currently on my macbook i have a tiny 128gb ssd :( with the bulk of it taken up by a single application, xcode. this issue is not unique to me, as it seems applications just keep ballooning in size.
so, if i don't have enough storage capacity, and extra ssd capacity is too expensive, why not use a cheaper hard disk drive instead?. first off, apple does not provide hdd options for their macbooks and secondly, even if i did have the option to switch to a hdd, i wouldn't. for the sole reason that ssd's are just superior in every relevant category to hdd's, and the most important factor for me is speed, and hard disks do not even come close when compared with ssds, as ssds are able to copy a 20gb movie in less than 10 seconds, whilst the same task would require at least 2 minutes with a hdd, so in all honesty hdd's have never really been an option for me as they are just too slow. but as i continued to use my mac, storage capacity became so much of an issue that i decided to base my final year dissertation on creating a system that would allow anyone to offload files from their computer and onto a cloud storage service, but before i reached the point of deciding to create a new system, i did some research to figure out how i could solve my storage problem, and at the end of it i decided against using currently available technology, such as external drives or cloud storage services (i hadn't considered decentralised storage solutions because i didn't know what they were at the time), with the main reason being that, unlike these existing offerings, the system i had come up with would cost 0£, so i decided it would be worthwhile building it if, at the end, i get a large amount of storage capacity, virtually for free.
the solution is actually quite simple. it involves having a group of users pool a predetermined amount of storage capacity available on their device into a single shared resource where anyone can upload and share their files. so you could commit just 10gb of your own storage capacity, and in return, you could get 100gb/200gb/500gb/however much the total shared storage capacity would come to be, all for free.
in more technical terms, the system would work as a p2p system where the metadata about the uploaded files and the details about the nodes hosting them would be stored on a distributed blockchain. simple right :). with this, you lose the need for central servers (which cost money) by virtue of the entire network being p2p, and you lose the need for a hosted database (which also costs money) by having a blockchain that keeps track of every upload/delete event and finally by having the participants' store shards of the uploaded files with the nodes in the network(explained later) there is no need for any cloud data management system. so what you end up with is a system capable of performing most of the functions a traditional external storage or cloud storage system can, without the user having to pay a dime.
before i conclude i'll explain some concepts used in the system. the first being shards. before a file is uploaded it is first converted into byte data and then divided into partitions, and each partition is called a shard. once each shard has been created they are encrypted and sent to nodes known to the user uploading the file, then to download those files, each shard is retrieved from their hosts, decrypted, and then combined, to recreate the original file (because files are just a big byte array, the combination process is actually quite straightforward as each byte just needs to be placed in the correct positions to recreate the byte array of the original file, kind of like appending different words together to make a sentence). the next concept is the type of blocks. unlike most blockchain systems which have just 1 type of block, this system has 2. the first being an upload block (created when the user uploads a block) and a delete block (created when the uploader of a file deletes it).
next is how users would be incentivised to actually stay on the network and use it. given that there are no central servers, the availability of files is totally dependent on the ability of the users to maintain uptime, as they host the shards required to recreate a file. unlike transaction data which gets generated after a transaction has occurred, files exist outside of blockchains, and so it would not make sense to award people with tokens when they create a new block as people could just upload junk files and receive tokens for it. the solution was in using both the amount of time a user had spent on the platform and the size of shards they had stored on their device as the criteria for awarding the tokens. initially, there is a maximum number of tokens from tokens are given to the user in increments over a regular cadence (e.g. they could receive 1 token every hour) while they are online (which gets wiped out when they log off), and once the user has then gotten all of those tokens, the only way to get more would be by storing shards on their device. given that the tokens are not transferrable between devices, there is no way for a user to skip having to stay online and/or host the shards of other users on their device. the tokens received for hosting other shards would also be issued out at a regular cadence, i.e. if the user is hosting 1 gb of shard data, they would receive 1gb worth of shards every 10/20/30/?? mins as a result of it.
in conclusion, the system in its simplest form is able to provide the function of hosting files without the user having to pay anything, but there are still a lot of kinks to be worked out before it can become a viable product. there's also a lot that i did not go over, as the goal is not to rewrite my entire dissertation. the goal of this essay was to highlight that the idea of limitless storage was not impossible and that someday hopefully in the near future the idea of just ticking a box to opt-in to pooled storage for free becomes possible. i should add that this is not a new idea and a lot of people are already working on this problem. you can check out ipfs, filecoin, sia if you're interested in this.
👋🏿