Asynchronous HTTP significantly speeds up operations involving multiple
JSON-RPC calls to the server, such as tracking wallet views for wallets
with a large number of outputs.
This patch adds base-level asyncio infrastructure plus aiohttp support to all
applicable MMGen commands.
The aiohttp package is not currently supported by MSYS2, so Windows users will
have to choose one of the other backends ('curl' is the default).
Tested on: Linux, Armbian, Windows; Python 3.6, 3.7, 3.8
New user features:
- configurable RPC backends via the 'rpc_backend' option. Supported
options are 'aiohttp' (Linux-only), 'httplib', 'requests' and 'curl'
- configurable RPC queue size via the 'aiohttp_rpc_queue_len' option
The patch also includes a rewrite/redesign of large parts of the MMGen code
base, most importantly:
- rpc.py - full rewrite of RPC library, new RPCBackends class
- main_addrimport.py - full rewrite
- main_autosign.py - LED code now handled by new LEDControl class
- eth/tw.py, eth/tx.py - reworked logic for resolving token symbols and
addresses
- eth/tx.py - separate classes for signed and unsigned transactions
Testing:
# Set a backend (choose one):
$ export MMGEN_RPC_BACKEND='aiohttp' # Linux-only
$ export MMGEN_RPC_BACKEND='curl' # Windows
$ export MMGEN_RPC_BACKEND='httplib' # compare performance with 'aiohttp'
# Bitcoin:
$ test/unit_tests.py rpc btc
$ test/test.py main regtest autosign
# Ethereum:
$ test/unit_tests.py rpc eth
$ test/tooltest2.py --coin=eth --testnet=1 txview
$ test/test.py --coin=eth ethdev
# Monero wallet:
$ test/unit_tests.py rpc xmr_wallet
$ test/test-release.sh -F xmr
Auto-completion functionality for seed phrase entry provides real benefit to the
user, reducing the number of keystrokes required and permitting quick re-entry
of mistyped words. In addition, unifying the number of keystrokes among words
improves security against acoustic side-channel attacks. To this end, three
new interactive mnemonic entry modes are introduced by this patch.
Each entry mode is optimized for a particular wordlist. The “short” mode, for
example, takes advantage of the fact that each word in the Monero wordlist is
uniquely identifiable by its first three letters. For MMGen’s default Electrum
wordlist, which lacks this unique substring property, the “minimal” mode was
developed to reduce keystrokes to a minimum while retaining the option of
obfuscating entry with pad characters.
Users who prefer not to use auto-completion may specify the “full” mode, which
emulates the previous default behavior.
Overview of the key entry modes:
- 'full' (all wordlists): words are typed in full and entered with the ENTER
or SPACE key, or by exceeding the pad character limit (see below).
- 'short' (BIP39, Monero): words are entered automatically once user types
UNIQ_SS_LEN (see below) valid word letters. 3-letter words in the BIP39
wordlist must be entered with the ENTER or SPACE key, or by exceeding the
pad character limit.
- 'fixed' (BIP39, Electrum): words are entered automatically once user types
UNIQ_SS_LEN characters in total. Words shorter than UNIQ_SS_LEN must be
padded to fit. Thus the total number of characters entered is the same for
all words.
- 'minimal' (Electrum): words are entered automatically once user types the
minimum number of characters required to uniquely identify a word (varies
from word to word). Words that are substrings of other words in the wordlist
must be entered with the ENTER or SPACE key, or by exceeding the pad
character limit. This is the only mode that checks user input letter by
letter.
Pad character limits by mode:
-----------------------------
short: 16
minimal: 16
full: longest_word - word_len
fixed: uniq_ss_len - word_len
Wordlist parameters:
--------------------
Parameter Electrum BIP39 XMRSEED
--------- -------- ----- -------
uniq_ss_len: 10 4 3
shortest_word: 3 3 4
longest word: 12 8 12
optimum mode: minimal fixed short
Default modes for each wordlist may be configured in 'mmgen.cfg' via the
'mnemonic_entry_modes' option.
Usage / testing:
$ mmgen-walletconv -i words
$ mmgen-walletconv -i bip39
$ mmgen-tool mn2hex_interactive fmt=mmgen mn_len=12 print_mn=1
$ mmgen-tool mn2hex_interactive fmt=bip39
$ mmgen-tool mn2hex_interactive fmt=xmrseed
$ test/unit_tests.py mn_entry
$ test/test.py -e input
- only 256-bit (25-word) new-style mnemonics are supported
Testing:
$ test/unit_tests.py baseconv
$ test/tooltest2.py hex2mn mn2hex
$ test/scrambletest.py pw
$ test/test.py ref_xmrseed_25_passwdgen_3
$ test/test.py ref_passwdfile_chk_xmrseed_25
The following operations are supported:
Generate a random Monero mnemonic:
$ mmgen-tool mn_rand256 fmt=xmrseed
Generate a Monero mnemonic from hexadecimal data:
$ mmgen-tool hex2mn deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef fmt=xmrseed
Convert the resulting mnemonic back to hexadecimal data:
$ mmgen-tool mn2hex 'viewpoint donuts ardent template unveil agile meant unafraid urgent athlete rustled mime azure jaded hawk baby jagged haystack baby jagged haystack ramped oncoming point template' fmt=xmrseed
Note that the result of the reversal does not match the original input. This
is because input data is reduced to a spendkey before conversion so that a
canonical seed phrase is produced. This is required because Monero seeds,
unlike ordinary wallet seeds, are tied to a concrete key/address pair. The
spendkey can be generated directly using the `hex2wif` command:
$ mmgen-tool --coin=xmr hex2wif deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef
Generate a list of passwords in Monero mnemonic format with ID 'mymonero':
$ mmgen-passgen -f xmrseed 'mymonero' 1-10
- Permits the creation of wallets by repeated rolls of a die: 50 rolls for
128-bit, 75 for 192-bit, and 100 for 256-bit seeds. The base6d format
uses the digits from one to six, so that user doesn't have to subtract
one from each roll.
Testing:
$ test/unit_tests.py baseconv
$ test/test.py ref3 conv
Example:
NOTE: when creating a real wallet, the following steps must be performed in a
secure offline environment, and preferably with the use of a text editor and
a file written in volatile memory (e.g. /dev/shm), rather than the `echo`
command. A more private and user-friendly data input method will be provided
in a forthcoming patch.
Sample 128-bit data obtained by rolling a die 50 times and entering each roll
on the keyboard as a digit:
15146 56446 53415 45431 55141 32115 41325 16311 32553 43533
Here spaces have been added for greater readability. Newlines are also
permitted.
Save the data in a file with the extension .b6d:
$ echo 15146 56446 53415 45431 55141 32115 41325 16311 32553 43533 > myseed.b6d
Convert to MMGen's default wallet format:
$ mmgen-walletconv -o wallet myseed.b6d
Convert the wallet back to base6d:
$ mmgen-walletconv -o dieroll FE3C6545*.mmdat
Base6d die roll seed data written to file 'FE3C6545[128].b6d'
$ cat 'FE3C6545[128].b6d'
15146 56446 53415 45431 55141
32115 41325 16311 32553 43533
- provided as an alternative to MMGen's native mnemonic format
# Run the BIP39 unit test:
$ test/unit_tests.py -v bip39
# Generate a random 128-bit BIP39 seed phrase:
$ mmgen-tool mn_rand128 fmt=bip39
# Export your default wallet to BIP39 format:
$ mmgen-walletconv -o bip39
...
BIP39 mnemonic data written to file '98831F3A[256].bip39'
# Generate ten addresses from the exported wallet:
$ mmgen-addrgen '98831F3A[256].bip39' 1-10
...
Addresses written to file '98831F3A[1-10].addrs'
# Generate ten addresses directly from your BIP39 seed phrase:
$ mmgen-addrgen -q -i bip39 1-10
...
Addresses written to file '98831F3A[1-10].addrs'
# Export subwallet 10L of your default wallet to BIP39 format:
$ mmgen-subwalletgen -o bip39 10L
...
BIP39 mnemonic data written to file 'A17F8E90[256].bip39'
- create multiple seed splits with a single “master share”. The master share
is a deterministic pseudorandom value which is HMAC’ed with the split’s ID
string and split count to produce the first share of the split. To ensure
uniqueness between splits, the remaining pseudorandom shares have the master
share’s index appended to their HMAC key. Each seed has 1024 numerically
indexed master shares.
