# FIP: Reduce Storage Limits Source: https://github.com/farcasterxyz/protocol/discussions/189 Kiwi News: https://news.kiwistand.com/stories/FIP-Reduce-Storage-Limits?index=0x668d5fd811456e8eb7cc3918367d756a5e1c47adca1836021b49a6ba1997b829c5f69136 Submitted by tim on 2024-07-09 UTC, 1 upvote, 0 comments ## Summary Farcaster proposes lowering per-unit storage limits for casts, reactions and links without changing storage unit prices, to slow the growth of hub data. Hub data is about 140GB today and grows 1GB/day, with a soft cap at 7 TB where high-performance NVMe disks tap out. Casts would drop from 5000 to 4000, and reactions and links from 2500 to 2000, an estimated 5% reduction in hub data size. About 1% of accounts, mostly programmatic ones, would lose casts and links, and about 5% would lose reactions; the new limits take effect 2024.08.08, one week after protocol version 2024.07.24. ## Article Authors @sanjayprabhu @varunsrin @Wazzymandias Problem A hub stores a copy of all data on the Farcaster network. This is about 140GB today and is growing at 1GB/day. This is problematic for a few reasons: Increasing data size will lead to slower startup and sync processes. There is a soft cap at 7 TB — high performance hubs need NVMe disks which tap out here. Farcaster’s growth is unpredictable and could 10x quickly creating new problems. Proposal We propose reducing the amount of space that each storage unit provides without changing the price. The new limits for casts, reactions, and links will be: Type Old Limit Proposed Limit Users Impacted Casts 5000 4000 ~1700 Reactions 2500 2000 ~23000 Links 2500 2000 ~5700 UserData 50 50 No Change Verification 25 25 No Change Username Proof 5 5 No Change Analysis Casts, links, and reactions use roughly 70% of all storage space for users on hubs The average user has around 179 casts, 418 reactions, and 236 links Message Type Count Total Bytes Total Bytes % Cast 92.0M 23.6B 18.38 Link 121.3M 22.4B 17.47 Reaction 214.7M 43.5B 33.94 Verification 411.8K 118.3M 0.09 UserData 2.0M 406.9M 0.32 Username Proof 10.2K 2.905M 0.00 Impact These changes should decrease the size of data on the Hub by 5%. A few accounts (~ 1%) have casts and links that will be pruned by this change. A majority of these accounts are “programmatic” and post automated replies (e.g. @perl). They can easily resolve this by buying another storage unit. A slightly larger number (~ 5%) have reactions that will be impacted by this. A majority of these expired reactions are not useful to the network. Keeping them around makes hubs more difficult to operate while providing little value to the network. Anyone who really wishes to retain more reactions can always purchase more storage. Rationale Why not increase the price of storage units? Making storage more expensive will reduce the number of users signing up. This isn’t good because real users are price sensitive while airdrop farmers aren’t. Can Hubs be sharded to scale better? Yes, but sharding makes sync and availability orders of magnitude more complex for developers. While we will eventually need to do this, we want to avoid it until we are closer to the 7 TB cap. Can we use compression or other lossless optimizations? We’ve explored compression, removing indices and other such changes. The best ones we’ve found reduce 1% — 2% of storage but come with a lot of implementation complexity. The proposed change is much simpler and saves much more storage. Do we have to do this now? Farcaster’s growth is unpredictable — we’d rather propose and implement these changes when we have the time and space for a discussion, instead of doing it in an emergency situation./ Will this impact channel moderation? Channel moderation depends on likes, which are getting more expensive. If you are running a channel bot that will be affected by this, reach out to @dwr.eth or @v. Warpcast will sponsor a few storage units for channel bots to make this transition simpler. I use all my reaction storage but very little of my cast storage. Can storage units be flexible? Flexible storage units are very tricky to implement with a good UX. There are many problems, but the most pressing one is what to do when you’ve exceeded the limit and send a new message. The only solution with our eventually consistent network is to prune the oldest message, irrespective of type. This means that if you run out of space and like something, the first person you followed may get unfollowed, which is really bad. There is no easy solution to this problem. Release This change will be part of the next protocol version 2024.07.24 . Since the change is backwards incompatible, the new limits will not be enforced until 2024.08.08, one week after protocol version release. Pruning happens every two hours so we expect that your hub will enforce the new limits for its users within that period. Hubs may be slightly out of sync for a few hours until the pruning processes are complete. Users and accounts that want to preserve data should upgrade their storage before this date.