Goal: for each test run to produce reproducible output, allowing us to check
for possible stochastic behavior in the scripts, as well as output-related
regressions (for example, garbage or improperly formatted output produced by a
bad format string) that might not be detected by the test scripts.
In practice, bugginess of the pexpect module and the non-deterministic behavior
of Bitcoin Core’s regtest implementation preclude completely identical output
from test run to test run, but the differences are small enough to result in an
easily reviewable diff.
Enable this feature by setting the MMGEN_TEST_SUITE_DETERMINISTIC environment
variable or running test/test-release.sh with the -D switch.
Examples:
$ script -c 'test/test-release.sh -FDv quick' -O run1
$ script -c 'test/test-release.sh -FDv quick' -O run2
# (optionally remove control characters from output files)
$ diff -u run1 run2 > diff
$ export MMGEN_TEST_SUITE_DETERMINISTIC=1
$ script -c 'test/test.py -ne main' -O run1
$ script -c 'test/test.py -ne main' -O run2
# (optionally remove control characters from output files)
$ diff -u run1 run2 > diff
Rationale of this commit: to relocate some ugly test-related code from the MMGen
package tree to the test tree, as well as to enable deterministic testing
(implemented in the next commit).
The overlay tree is a symlinked mirror of the MMGen package dir with a few
monkey-patched modules.
The monkey-patching is conditional, so the modules are certain to get tested in
their unpatched state as well.
- During initialization, data objects now invariably raise an exception on
failure (ObjectInitError by default, configurable via the 'exc' attribute).
- For callers that need to handle the exception, the new get_obj() wrapper is
provided.
Testing:
$ test/objtest.py -S
$ test/objtest.py -S --getobj
This patch eliminates nearly all the global variables that changed during the
execution of scripts. With a few minor exceptions, global vars are now used
only during initialization or reserved for cfg file / cmdline options and other
unchanging values.
The result is a code base that's much more maintainable and extensible and less
error-prone. The autosigning code, which supports signing of transactions for
multiple protocols and networks, has been greatly simplified.
Doing away with globals required many changes throughout the code base, and
other related (and not so related) changes and cleanups were made along the
way, resulting in an enormous patch.
Additional code changes include:
- tx.py: complete reorganization of TX classes and use of nesting
- protocol.py: separation of Regtest and Testnet into distinct subclasses
with separate address and transaction files and file extensions
- new module help.py for the help notes, loaded on demand
- addr.py: rewrite of the address file label parsing code
- tx.py,tw.py: use of generators to create formatted text
User-visible changes include:
- importing of addresses for tokens not yet in the user's tracking wallet
is now performed with the `--token-addr` option instead of `--token`
Testing:
Testing this patch requires a full run of the test suite as described on the
Test-Suite wiki page.
Beginning with this commit, every MMGen wallet now has a two sets of associated
subwallets with “long“ and “short” seeds.
MMGen wallets and subwallets are functionally equivalent and externally
indistinguishable. This has benefits, especially for real-world security, as
well as drawbacks. For more information, see the `mmgen-subwalletgen` help
screen: https://github.com/mmgen/mmgen/wiki/subwalletgen-[MMGen-command-help]
This patch provides subwallet generation functionality and subseed display
utilities. Support for transaction signing and address generation using a
subwallet's parent wallet will be added in a forthcoming patch.
Examples:
# Create a bogus wallet in mnemonic format for testing purposes:
$ echo $(yes bee | head -n24) > bogus.mmwords
# List the wallet's first five subseed pairs:
$ mmgen-tool list_subseeds 1-5 wallet=bogus.mmwords
Parent Seed: DF449DA4 (256 bits)
Long Subseeds Short Subseeds
------------- --------------
1L: FC9A8735 1S: 930E1AD5
2L: 62B02F54 2S: DF14AB49
3L: 9E884E99 3S: AD3ABD98
4L: DB595AE1 4S: 3E885EC4
5L: 36D5A0D1 5S: 30D66FF5
# Generate the 5th short (128-bit) subwallet from the wallet:
$ mmgen-subwalletgen bogus.mmwords 5S
# Same as above, but output subwallet to mnemonic (seed phrase) format:
$ mmgen-subwalletgen -o mn bogus.mmwords 5S
...
Mnemonic data written to file '30D66FF5[128].mmwords'
# View the subwallet's seed phrase:
$ cat 30D66FF5[128].mmwords
object capture field heart page observe road bond mother loser really army
# Generate 10 addresses from the subwallet seed phrase:
$ mmgen-addrgen 30D66FF5[128].mmwords 1-10
...
Addresses written to file '30D66FF5[1-10].addrs'
- When parsing opts, opts.init() now looks only at string values from
opts_data. Global variables are evaluated only when printing help text,
after the variables are initialized
- binascii.hexlify(b'foo') -> b'foo'.hex()
- binascii.unhexlify('aabb') -> bytes.fromhex('aabb')
- replace HexBytes class with HexStr
This change has led to a ≈10% speedup in the full test-release.sh run
This is the first commit of the MMGen Python 3 port. Branch 'py3port', against
master branch commit ab06ca4
Commits are groupings of similar or related changes: automatic changes first,
followed by repetitive and global changes, followed by specific ones
All commits until the final one will leave the branch in a broken state
- view keys are included in key-address lists
- tests updated to test the new functionality
- mmgen-tool: `compressed` and `segwit` args replaced by --type option
- basic data types in obj.py rewritten
- new test suite 'test/objtest.py' for testing basic data types
- new compressed address type with the 'C' identifier
- Bob and Alice regtest mode for testing MMGen in a mock two-user environment
* All MMGen commands are available in this mode.
* Set up with 'mmgen-regtest setup'. Bob and Alice's wallets are funded with
500 BTC each. Use the --mixed switch to import mixed address types in
Bob and Alice's tracking wallets.
* Transact as Bob by adding --bob switch to MMGen commands
* Transact as Alice by adding --alice switch to MMGen commands
* After sending a transaction, mine a block to confirm it with
'mmgen-regtest generate'
The bitcoin daemon is stopped and restarted automatically when switching
between users.
