Draft: start working on a nesting-netDB design, where it is possible to segment the netDB
start working on a nesting-netDB design, where it is possible to segment the netDB.
Proposal to allow Java I2P to Host Multiple NetDB's
This outlines a proposed way to allow Java I2P to host multiple "contexts" of NetDB, similar to i2pd.
Current situation
Java I2P has existed for 20 years, and during that time it has experienced several revisions to it's NetDB. However, throughout most of that time, there has only ever been one NetDB, managed in the Router context, which is queried for data. This one-NetDB approach requires, at various layers, that the NetDB keep track individually what RouterInfos and LeaseSets are acceptable to send or store, and when. Currently, the Java I2P NetDB does this by maintaining a set of "Flags" on each "DatabaseEntry." These flags indicate the circumstances under which a LeaseSet or RouterInfo has been encountered. Without these, the router would be vulnerable to attackers who submit crafted LeaseSets in order to "Tag" clients and routers for context-confusion attacks. This is extremely complicated to maintain and keep track of the validity of any given NetDB operation, and has led to difficult to predict and difficult to fix attacks.
Proposed Solution
Background: The solution adopted in i2pd, and often lauded by those who study it, is to give each potential context in which the NetDB might be required it's own NetDB to communicate with, treating each NetDB as an entirely separate entity with each context. The simplicity of this solution has great appeal, but it is almost antithetical to the incumbent design of the NetDB in relation to the router context.
In order to maintain the existing functionality of the Java I2P router while also providing a way of interfacing
with multiple NetDB's to avoid context confusion attacks, a global "Meta-NetDB" will replace the existing NetDB,
which will consist of an interface to manage multiple Sub-NetDB's. The outer, "Meta-NetDB" is a "SegmentedNetworkDatabase
,"
and the inner, "Sub-NetDBs" are a string-addressed map of "FloodfillNetworkDatabase
s."
Goals of the proposed solution
- Don't Break Anything - make no mistake, this could break everything,.potentially, because it's going to affect everything that touches the NetDB, which is everything. The potentially impacted places run from pages in the router console UI to NetDB on-disk persistence to the viability of encrypted leaseSets. This is the hardest thing to do while also fixing the problem.
- Make context-confusion attacks more difficult by providing differing NetDB contexts as-neeeded - Contexts will be necessary for clients and floodfills, but they might also be necessary for exploratory clients and potentially other parts of the router.
- Preserve the ability to communicate with the NetDB globally via the Meta-NetDB - Especially in router console pages, it will be important to talk to all of the NetDB contexts.
- Leave Existing Defenses Intact - Using the existing defenses is still viable to an extent, we're not going to remove them. It's more work for one thing, and for another we can use the existing defenses to track the consistency where the new defenses are applied.
Details of the Proposed Solution
- A new abstract Class is created, SegmentedNetworkDatabaseFacade
- A new implementation FloodfillNetworkDatabaseSegmentor
- In RouterContext, getDb() is changed to return the global FloodfillNetworkDatabaseSegmentor
- In RouterContext, new functions floodfillNetDB() and clientNetDB(), clientNetDB(string) return FloodfillNetworkDatabaseFacade
- Clients must use clientNetDB(string) call to obtain a private context, where the string is the key used to look them up in the map
- Floodfills must use the floodfillNetDB() call to obtain the NetDB context for acting as a floodfill
- Local displays(such as on the routerconsole) will need to call getDb() to get the global NetDB context and query all peers
Merge request reports
Activity
assigned to @idk
added 24 commits
-
fae11a0a...eb0c8aaa - 4 commits from branch
i2p-hackers:master
- eb0c8aaa...f23ac402 - 10 earlier commits
- 12c1c945 - don't call the function just check the var
- 646d2623 - in HandleDatabaseLookupMessage always treat LeaseSets stored in the...
- 61abf49f - fix typo, deduplicate throttler code, log potentially-bannable bursts earlier...
- e912f20b - adjust the timers of the lookup throttlers
- c2e8a971 - update logging messages
- 8805bf09 - small tweak to how stores work
- e6c74fb4 - It would seem that it is not a reliable safety check to prevent setRecievedAsReply
- e26754f9 - do the flag updates after the overwrite
- fa7b3131 - serve leaseSet out of possibleMultihomed if we are not closest in the keyspace
- 0327adc2 - Fix merge conflict
Toggle commit list-
fae11a0a...eb0c8aaa - 4 commits from branch
added 15 commits
-
0327adc2...206cc003 - 14 commits from branch
i2p-hackers:master
- 8f5a9a0d - Start using prototype of multi-netDB to find uses of single-netDB.
-
0327adc2...206cc003 - 14 commits from branch
added 1 commit
- 711f34d4 - Router: Segmented NetDB/NetDB Compatibility progress
TODO LIST:
Identify every place in the router console where we call
getDb()
and what it's used for.Identify every place in the router and/or apps where a CLIENT references
getDb()
.Identify every place in the router and/or apps where
getDb()
is called by a FLOODFILL.- LIST "FLOODFILL CALLS" HERE
- every place where we now call
floodfillEnabled()
right now should be on thefloodfillNetDb()
Identify places where an encrypted leaseSet must do something with a blinding key or a leaseSet encryption key and determine if they are client calls or floodfill calls(They are likely mostly client calls with isolating keys)
Edited by idkadded 1 commit
- d04ddefc - Router: begin auditing netDb() calls in detail, in 'RouterInfoHandler'...
added 1 commit
- 4c58041e - but always call startup when startup is called by the metanetdb
added 1 commit
- d6b3a6e2 - Console: Add menu items for viewing client NetDB's into NetDbHelper
added 1 commit
- 416f9a5b - Console: allow to get only routers known to clients subNetDBs when viewing the netDB
added 4 commits
Toggle commit listadded 1 commit
- bd557e37 - Router: fix SO in FNDF caused by improper recursion, call super not this
added 1 commit
- e1a79c2c - Router: fix SO in FNDF caused by improper recursion, call super not this
added 1 commit
- c137b0f0 - Router: Migrate more functions to the new Segmented netDb. This checkin is...
added 1 commit
- ad2a58c3 - Router: Expansive changes to netDb handling which enable the use of the 'Safe'...
added 1 commit
- 94e36a1e - Router: use client's blind-caches for blinding keys. Note that I don't think...
added 1 commit
- 9cc2b24a - Router: this version updates all blinding keys when a blinding key appears in more than one netDb
added 1 commit
- 7bc50044 - Router: seed the RI pool for subDbs with a more reasonable number of routerInfos
added 1 commit
- 5e603578 - Router: add TODO note to ConfigKeyringHandler
added 1 commit
- fa2fc67f - Router: log the b32 on the line before the ttps in InboundMessageDistributor...
added 1 commit
- ab3c919e - Router: make the minimum number of floodfill peers where the subdb is seeded adjustable
added 1 commit
- 6427efac - Router: delete the sub-netDb when the session is destroyed
added 1 commit
- abc52847 - Router: delete the sub-netDb when the session is destroyed
added 1 commit
- 42f2dd03 - Router: make sure the sub-netDb's use the correct peerSelector. Format the code in Segmentor.
added 1 commit
- ecb3e2a1 - Router: Don't publish local RI unless dbid is floodfill
added 1 commit
- e3d6e225 - Console: Resolve some of the netDb rendering issues on .netdb
added 1 commit
- 8a250344 - Console: don't re-render our own RI over on /netdb
Cross-reference !101 (merged)
mentioned in merge request !101 (merged)