Goal: for each test run to produce reproducible output, allowing us to check
for possible stochastic behavior in the scripts, as well as output-related
regressions (for example, garbage or improperly formatted output produced by a
bad format string) that might not be detected by the test scripts.
In practice, bugginess of the pexpect module and the non-deterministic behavior
of Bitcoin Core’s regtest implementation preclude completely identical output
from test run to test run, but the differences are small enough to result in an
easily reviewable diff.
Enable this feature by setting the MMGEN_TEST_SUITE_DETERMINISTIC environment
variable or running test/test-release.sh with the -D switch.
Examples:
$ script -c 'test/test-release.sh -FDv quick' -O run1
$ script -c 'test/test-release.sh -FDv quick' -O run2
# (optionally remove control characters from output files)
$ diff -u run1 run2 > diff
$ export MMGEN_TEST_SUITE_DETERMINISTIC=1
$ script -c 'test/test.py -ne main' -O run1
$ script -c 'test/test.py -ne main' -O run2
# (optionally remove control characters from output files)
$ diff -u run1 run2 > diff
Rationale of this commit: to relocate some ugly test-related code from the MMGen
package tree to the test tree, as well as to enable deterministic testing
(implemented in the next commit).
The overlay tree is a symlinked mirror of the MMGen package dir with a few
monkey-patched modules.
The monkey-patching is conditional, so the modules are certain to get tested in
their unpatched state as well.
This patch eliminates nearly all the global variables that changed during the
execution of scripts. With a few minor exceptions, global vars are now used
only during initialization or reserved for cfg file / cmdline options and other
unchanging values.
The result is a code base that's much more maintainable and extensible and less
error-prone. The autosigning code, which supports signing of transactions for
multiple protocols and networks, has been greatly simplified.
Doing away with globals required many changes throughout the code base, and
other related (and not so related) changes and cleanups were made along the
way, resulting in an enormous patch.
Additional code changes include:
- tx.py: complete reorganization of TX classes and use of nesting
- protocol.py: separation of Regtest and Testnet into distinct subclasses
with separate address and transaction files and file extensions
- new module help.py for the help notes, loaded on demand
- addr.py: rewrite of the address file label parsing code
- tx.py,tw.py: use of generators to create formatted text
User-visible changes include:
- importing of addresses for tokens not yet in the user's tracking wallet
is now performed with the `--token-addr` option instead of `--token`
Testing:
Testing this patch requires a full run of the test suite as described on the
Test-Suite wiki page.
- When parsing opts, opts.init() now looks only at string values from
opts_data. Global variables are evaluated only when printing help text,
after the variables are initialized
This is a work in progress. Currently, basic operations for BTC and ETH are
supported.
The successor to the MinGW64 project, MSYS2 features package management via
`pacman` and support for Python 3:
https://sourceforge.net/projects/msys2https://www.msys2.org
The rationale for this patch is similar to that of commit a7126ed:
- Many packages were imported for the sake of just a few trivial conversion
functions. These functions have been copied into the local pyethereum.utils.
- rlp has been locally copied and its import statements modified to import
the locally-copied functions. Unneeded classes and functions have been
removed.
- As a result, dependencies on the following external packages have been
eliminated:
+ rlp
+ eth-hash
+ eth-utils
+ eth-typing
+ toolz
+ cytoolz
+ setuptools
- pysha3: Monero and Ethereum use the old pre-SHA3 version of the keccak
hash function, which is not supported by hashlib.sha3.
The pysha3 package supports the function but is not portable.
Therefore, use the pure-Python implementation mmgen.keccak as a fallback,
making the pysha3 package optional.
Use of mmgen.keccak may be forced with --use-internal-keccak-module
mmgen.keccak was ported from the Python 2 implementation at
https://github.com/ctz/keccak
- ethereum (pyethereum): Installation of this package presents numerous
problems due to poor maintenance and many superfluous dependencies.
Therefore, use stripped-down and modified local versions of
ethereum.transactions and ethereum.utils as fallbacks, making the ethereum
package optional.
The local pyethereum.utils uses mmgen.keccak as a fallback for pysha3's
keccak implementation.
tooltest.py - bugfixes, remove some commands covered in tooltest2.py
mmgen-tool - bugfixes, cleanups, rename some commands, change some command
options
- all commands taking binary input can now receive it from file
or stdin
+ numerous minor fixes throughout
- test groups are now separate classes in separate modules
- test data and code is loaded on an as-needed basis
- new TestSuiteRunner and CmdGroupMgr classes
- simplified invocation: if arguments are omitted, all default tests relevant
for given network and option are run. The following set of invocations
provides nearly complete coverage of MMGen's core functionality:
test/test.py
test/test.py --segwit-random
test/test.py --bech32
test/test.py --coin=ltc
test/test.py --coin=ltc --segwit-random
test/test.py --coin=ltc --bech32
test/test.py --coin=bch
test/test.py --coin=eth
test/test.py --coin=etc
- group tool commands into classes
- add docstrings to command classes and methods
- annotate command method args
- generate call signatures + help and usage screens on the fly
- API changes:
- listaddresses,twview: new 'age_fmt' option replaces 'show_days' and 'show_age'
- addrfile_chksum and friends: 'mmtype' option removed
- tooltest.py, tooltest2.py and test.py have been updated accordingly
