First Solar System problems!
Random Thoughts on Scalability
hyc posted this 8 years ago
It occurred to me today that scalability and anonymity are fundamentally opposed. In particular, scalability is about improving efficiency, but the technologies we're adding to Monero (e.g., Kovri/i2p) are inherently inefficient. We don't know if Monero will really be the currency of the future, but I know for certain, any coin that doesn't have inherently scalable technology won't be the currency of the future.
For example, we have Monero and Bitcoin emission curves plotted out to the year 2050 or so. There are multiple projects around the world dedicated to setting up permanent human colonies on Mars by the 2030s. https://en.wikipedia.org/wiki/Human_mission_to_Mars To me this means the currency of the future must scale to interplanetary scope, otherwise it's dead.
There are number of fundamental concepts in currently known cryptocoins that must be changed in order to make scaling feasible. At the heart of the problem is the blockchain itself - it's a linear data structure. I've never understood this continuous cognitive dissonance in the data management world - it's well understood that linear algorithms and structures can't scale. Every data management system in existence today uses some type of tree structure for indexing, because trees do scale, and are pretty much the only thing that does. And yet, both the SQL world and the NoSQL world continue to logically organize their data linearly. Tables of linear rows and columns for SQL, or just flat linear namespaces for NoSQL. This is idiocy of the highest order. We have plenty of examples of scalable tree structures embedded in our computing environment - DNS is hierarchical, all of your computer filesystems uses hierarchical directories, etc. And yet, with the notable exception of LDAP, we still aren't using hierarchical logical structures for our databases. This is broken.
One of the ideas kicking around in my head is to start sharding the "blockchain" into a "blocktree". We know that in Monero all key images must be preserved forever, to prevent double-spending. But there's no reason every node must store a complete copy of all key images. We could break them down into TRIEs, splitting at key prefixes to distribute portions of the key images around the network. (E.g., for a 256bit hash, use the 1st 8 bits to select 1 of 256 sets of seed nodes. Each of those seed nodes can carry its own set of 256 more seed nodes, based on the 2nd 8 bits of the hash, etc. etc.)
The other notion is to use address prefixing to partition the address space geographically. It would be analogous to IP subnetting. In this case, only nodes within the same geographical region as a receiving wallet address need to store the transactions for those addresses. And here we have an efficiency/anonymity conflict - this is the most efficient way to store the block data, but it means your anonymity set is much smaller. And if you happen to be a Martian colonist trying to pay for supplies sent from Earth, your total anonymity set will be only a couple hundred people...
Another aspect to all this is the network protocol. Sending a radio signal to Mars will take from 3 minutes (when the two planets are closest) to 21 minutes (farthest). This is too slow for an interactive-style network protocol. The best solution I can see to this is to use dedicated peer pairs to send blocks to each other. The Monero router on Earth would maintain a permanent list of the hashes of blocks it has sent to Mars. It would know which blocks the Mars node hasn't received yet, and just send them, without any Ihave/Sendme type of dialog in between. Likewise for the Mars to Earth direction. Notice that the 1-way propagation time is always longer than the Monero block emission time - so, if you don't use a partitioned multi-chain, you will always have blocks in flight that the other side doesn't know about yet, and one side or the other will never be able to converge on a consensus.
Ideally you send protocol data wrapped with a strong error-correcting code. Since retransmits for lost or damaged packets will take so long, you want to make sure any packets that you send are actually usable when they're received.
Really, there is no way to make any cryptocoin work on interplanetary scale if it continues to be built on linear blockchains. The coin of the future must use hierarchical data structures. And it must use batch-oriented network protocols that don't require immediate ACKs and replies.
Fundamentally, Monero anonymity relies on all addresses being equivalent and indistinguishable. Efficient network communication and storage requires addresses to be partitionable. I don't see any way to reconcile these two requirements.
The interplanetary stuff is silly. We (i.e. humanity) can't seem to stop screwing up earth's environment. How could we possibly create environment on Mars if we can't stop wrecking our own? I don't know what it would take to stop carbon dioxide emissions. Carbon capture probably won't scale (so i read), so I think we should focus on moving away from fossil fuels altogether and, maybe scientists can come up with a way to draw CO2 out of the atmosphere and then split it (using a technique like hidrolysis). This aside, colonising Mars would also require creation of a planetary ionsphere over Mars, both to stop the atmosphere from being gradually blown away by the solar wind and to shield life on the surface from radiation. If we could get an earthlike atmosphere on mars (in sufficient quantity to have earthlike pressure at the Martian surface), we would then have to heat the atmosphere somehow, otherwise it would be so cold that the water and carbondioxide would just snow out of it. So right now Mars is barely any more coloniseable then any randomly chosen area of empty space!
It's interesting to think what Mars would be like if these problems really could be solved. It would be in general darker than earth being both further from the sun and having a greater depth of atmosphere (needed to have earth-like pressure). The greater depth of atmosphere might also imply more cloud cover in general, but I'm not sure).
However your main point was:
Fundamentally, Monero anonymity relies on all addresses being equivalent and indistinguishable. Efficient network communication and storage requires addresses to be partitionable. I don't see any way to reconcile these two requirements.
Obviously you can't be a maximalist in every domain at once. Hopefully research will show us the golden path whereby these two domains can be optimially reconciled.
A KISS (keep it simple stupid) approach to the interplanetary problem could be side chains. Then there's no need to time-synchronize a network.
But in general I like the concept. Is this similar to what Iota was doing with its DAG?
For me the thought experiment always boils down to the scenario where person A, who's usually in geographic area X, travels to geographic area Y and wants to spend their Monero. Presumably, person A could submit a transaction T to the network from area Y, and the area Y nodes wouldn't really know what to do with the transaction (i.e., the output isn't in area Y's database, because they are sharded). However, some node in the network can validate it, so its a matter of T making it to a node in area X, and X's nodes somehow communicating to the nodes in Y that the transaction is good.