This patch eliminates nearly all the global variables that changed during the
execution of scripts. With a few minor exceptions, global vars are now used
only during initialization or reserved for cfg file / cmdline options and other
unchanging values.
The result is a code base that's much more maintainable and extensible and less
error-prone. The autosigning code, which supports signing of transactions for
multiple protocols and networks, has been greatly simplified.
Doing away with globals required many changes throughout the code base, and
other related (and not so related) changes and cleanups were made along the
way, resulting in an enormous patch.
Additional code changes include:
- tx.py: complete reorganization of TX classes and use of nesting
- protocol.py: separation of Regtest and Testnet into distinct subclasses
with separate address and transaction files and file extensions
- new module help.py for the help notes, loaded on demand
- addr.py: rewrite of the address file label parsing code
- tx.py,tw.py: use of generators to create formatted text
User-visible changes include:
- importing of addresses for tokens not yet in the user's tracking wallet
is now performed with the `--token-addr` option instead of `--token`
Testing:
Testing this patch requires a full run of the test suite as described on the
Test-Suite wiki page.
Asynchronous HTTP significantly speeds up operations involving multiple
JSON-RPC calls to the server, such as tracking wallet views for wallets
with a large number of outputs.
This patch adds base-level asyncio infrastructure plus aiohttp support to all
applicable MMGen commands.
The aiohttp package is not currently supported by MSYS2, so Windows users will
have to choose one of the other backends ('curl' is the default).
Tested on: Linux, Armbian, Windows; Python 3.6, 3.7, 3.8
New user features:
- configurable RPC backends via the 'rpc_backend' option. Supported
options are 'aiohttp' (Linux-only), 'httplib', 'requests' and 'curl'
- configurable RPC queue size via the 'aiohttp_rpc_queue_len' option
The patch also includes a rewrite/redesign of large parts of the MMGen code
base, most importantly:
- rpc.py - full rewrite of RPC library, new RPCBackends class
- main_addrimport.py - full rewrite
- main_autosign.py - LED code now handled by new LEDControl class
- eth/tw.py, eth/tx.py - reworked logic for resolving token symbols and
addresses
- eth/tx.py - separate classes for signed and unsigned transactions
Testing:
# Set a backend (choose one):
$ export MMGEN_RPC_BACKEND='aiohttp' # Linux-only
$ export MMGEN_RPC_BACKEND='curl' # Windows
$ export MMGEN_RPC_BACKEND='httplib' # compare performance with 'aiohttp'
# Bitcoin:
$ test/unit_tests.py rpc btc
$ test/test.py main regtest autosign
# Ethereum:
$ test/unit_tests.py rpc eth
$ test/tooltest2.py --coin=eth --testnet=1 txview
$ test/test.py --coin=eth ethdev
# Monero wallet:
$ test/unit_tests.py rpc xmr_wallet
$ test/test-release.sh -F xmr
Auto-completion functionality for seed phrase entry provides real benefit to the
user, reducing the number of keystrokes required and permitting quick re-entry
of mistyped words. In addition, unifying the number of keystrokes among words
improves security against acoustic side-channel attacks. To this end, three
new interactive mnemonic entry modes are introduced by this patch.
Each entry mode is optimized for a particular wordlist. The “short” mode, for
example, takes advantage of the fact that each word in the Monero wordlist is
uniquely identifiable by its first three letters. For MMGen’s default Electrum
wordlist, which lacks this unique substring property, the “minimal” mode was
developed to reduce keystrokes to a minimum while retaining the option of
obfuscating entry with pad characters.
Users who prefer not to use auto-completion may specify the “full” mode, which
emulates the previous default behavior.
Overview of the key entry modes:
- 'full' (all wordlists): words are typed in full and entered with the ENTER
or SPACE key, or by exceeding the pad character limit (see below).
- 'short' (BIP39, Monero): words are entered automatically once user types
UNIQ_SS_LEN (see below) valid word letters. 3-letter words in the BIP39
wordlist must be entered with the ENTER or SPACE key, or by exceeding the
pad character limit.
- 'fixed' (BIP39, Electrum): words are entered automatically once user types
UNIQ_SS_LEN characters in total. Words shorter than UNIQ_SS_LEN must be
padded to fit. Thus the total number of characters entered is the same for
all words.
- 'minimal' (Electrum): words are entered automatically once user types the
minimum number of characters required to uniquely identify a word (varies
from word to word). Words that are substrings of other words in the wordlist
must be entered with the ENTER or SPACE key, or by exceeding the pad
character limit. This is the only mode that checks user input letter by
letter.
Pad character limits by mode:
-----------------------------
short: 16
minimal: 16
full: longest_word - word_len
fixed: uniq_ss_len - word_len
Wordlist parameters:
--------------------
Parameter Electrum BIP39 XMRSEED
--------- -------- ----- -------
uniq_ss_len: 10 4 3
shortest_word: 3 3 4
longest word: 12 8 12
optimum mode: minimal fixed short
Default modes for each wordlist may be configured in 'mmgen.cfg' via the
'mnemonic_entry_modes' option.
Usage / testing:
$ mmgen-walletconv -i words
$ mmgen-walletconv -i bip39
$ mmgen-tool mn2hex_interactive fmt=mmgen mn_len=12 print_mn=1
$ mmgen-tool mn2hex_interactive fmt=bip39
$ mmgen-tool mn2hex_interactive fmt=xmrseed
$ test/unit_tests.py mn_entry
$ test/test.py -e input
To allow the user to verify that transaction outputs have not been altered
by a compromised online MMGen installation, print a list of non-MMGen output
addresses and amounts for all signed transactions after every autosign run.
With --full-summary, print a full view of each signed transaction instead.
- binascii.hexlify(b'foo') -> b'foo'.hex()
- binascii.unhexlify('aabb') -> bytes.fromhex('aabb')
- replace HexBytes class with HexStr
This change has led to a ≈10% speedup in the full test-release.sh run
- test groups are now separate classes in separate modules
- test data and code is loaded on an as-needed basis
- new TestSuiteRunner and CmdGroupMgr classes
- simplified invocation: if arguments are omitted, all default tests relevant
for given network and option are run. The following set of invocations
provides nearly complete coverage of MMGen's core functionality:
test/test.py
test/test.py --segwit-random
test/test.py --bech32
test/test.py --coin=ltc
test/test.py --coin=ltc --segwit-random
test/test.py --coin=ltc --bech32
test/test.py --coin=bch
test/test.py --coin=eth
test/test.py --coin=etc
