LBRY-Vault/electrum
SomberNight 570c0aeca3
build: make NSIS windows binary deterministic by changing the .ico file
see bitcoin/bitcoin@217208a36d

-----

A lot of time was wasted on this... over the years actually...

Some notes and rant here, for future reference.

During the initial effort to try to make binaries reproducible,
out of the three windows binaries being distributed (standalone, portable, setup),
only the first two were successfully made deterministic.
Later, we started to use Docker-based builds. At that point ThomasV and I
could reproducibly build the same setup/nsis exe but Travis kept building a different one.

Recently I have noticed that if I do two subsequent builds of the setup exe on
the same machine, adding a new file in contrib/build-wine/ between the builds,
then I get different binaries. Playing around with this a bit, it seems:
- other files that are in the same folder as contrib/build-wine/electrum.nsi affect the binary
- only files that are in exactly the same folder matter (not recursively)
- only filenames matter (not permission, owner, timestamps, or file contents)
To see the difference in the binaries, use vbindiff, and disable the compression done
by nsis (SetCompress off).
There is a ~48 byte diff near the very beginning of the "Uninstaller" section.
I am only guessing it is the uninstaller section based on the sizes of the sections
printed by nsis during the build.
I have downloaded the binary built by Travis, and the diff is consistent with this
(i.e. it's the same kind of diff that manifests if I change the filename of one of
the supposedly unrelated files).
Commenting out the "WriteUninstaller" line in .nsi fixes the issue. i.e. if no
uninstaller is created then the binary becomes deterministic.
Commenting out the "!define MUI_ICON" line in .nsi also fixes the issue.
At this point I remembered the above referenced commit by bluematt; which I had
thought we had already followed up on...
Replacing the .ico file fixes the issue.
Note that it's not actually clear what the exact requirements for the .ico file are.
Removing any of the layers in the image seems to introduce non-determinicity.
The new .ico file has layers with resolutions and properties the bitcoin.ico file has.

I guess NSIS must have strict requirements for the icon size, and if a given size icon is missing
it might be creating it itself?? And during the downscaling it uses a non-deterministic
algorithm that initialises some RNG from the directory listing (bauerj's guess somewhat adapted :D).
Just crazy.
2019-06-24 21:51:47 +02:00
..
gui build: make NSIS windows binary deterministic by changing the .ico file 2019-06-24 21:51:47 +02:00
plugins hw: allow bypassing "too old firmware" error when using hw wallets 2019-05-31 04:09:03 +02:00
scripts scripts: simplify quick_start 2019-02-28 20:26:30 +01:00
tests tests: new tests for bump_fee and rbf_batching 2019-06-20 22:42:50 +02:00
wordlist file reorganization with top-level module 2018-07-13 14:01:37 +02:00
__init__.py logging: cli options to filter for modules using -v 2019-05-02 15:19:10 +02:00
address_synchronizer.py json_db: fix remove_spent_outpoint 2019-06-15 03:51:11 +02:00
base_crash_reporter.py kivy: fix crash in logging.py; platform.platform() not available 2019-05-06 19:10:29 +02:00
base_wizard.py hw: allow bypassing "too old firmware" error when using hw wallets 2019-05-31 04:09:03 +02:00
bip32.py wallet: stricter validation in export_private_key 2019-06-12 18:09:38 +02:00
bitcoin.py bitcoin: stricter check on WIF for compressed pubkeys 2019-04-25 14:35:16 +02:00
blockchain.py logging: basics 2019-05-02 15:19:03 +02:00
checkpoints.json update block header checkpoints 2019-02-08 16:38:59 +01:00
checkpoints_testnet.json update block header checkpoints 2019-02-08 16:38:59 +01:00
coinchooser.py coinchooser: clear up what "fee_estimator" expects 2019-06-20 22:46:22 +02:00
commands.py logging: don't log to file by default 2019-05-08 16:52:04 +02:00
constants.py bip32: refactor whole module. clean-up. 2019-02-22 18:50:24 +01:00
contacts.py logging: basics 2019-05-02 15:19:03 +02:00
crypto.py keystore: fail sooner if unsupported version 2018-12-18 19:57:58 +01:00
currencies.json exchange_rate: add Bylls BTC/CAD rates 2019-03-27 19:00:40 +01:00
daemon.py logging: basics 2019-05-02 15:19:03 +02:00
dnssec.py logging: basics 2019-05-02 15:19:03 +02:00
ecc.py commands: fix encrypt/decrypt 2019-05-03 03:10:31 +02:00
ecc_fast.py logging: basics 2019-05-02 15:19:03 +02:00
electrum file reorganization with top-level module 2018-07-13 14:01:37 +02:00
exchange_rate.py logging: basics 2019-05-02 15:19:03 +02:00
i18n.py move get_default_language to gui.qt.util 2018-09-23 14:11:50 +05:00
interface.py interface: hide some server-induced errors from log 2019-05-28 21:23:06 +02:00
json_db.py json_db: fix remove_spent_outpoint 2019-06-15 03:51:11 +02:00
jsonrpc.py logging: basics 2019-05-02 15:19:03 +02:00
keystore.py keystore/transactions: fix overflow of derivation path indices 2019-05-21 02:14:22 +02:00
logging.py logging: don't log to file by default 2019-05-08 16:52:04 +02:00
mnemonic.py logging: basics 2019-05-02 15:19:03 +02:00
msqr.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
network.py logging: '-V' cli option can blacklist/whitelist classes with short names 2019-05-07 21:07:18 +02:00
old_mnemonic.py mv "electrum seed" stuff from bitcoin.py to mnemonic.py 2019-02-22 18:01:54 +01:00
paymentrequest.proto file reorganization with top-level module 2018-07-13 14:01:37 +02:00
paymentrequest.py bip70 payreq: do not show error messages in gui 2019-06-05 19:40:33 +02:00
paymentrequest_pb2.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
pem.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
plot.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
plugin.py plugins: on some systems plugins with relative imports failed to load 2019-06-12 20:07:36 +02:00
qrscanner.py rename contrib/build-osx as contrib/osx. Move QRReader submodule there. 2018-11-29 11:39:57 +01:00
ripemd.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
rsakey.py no more "import *" 2019-02-11 20:21:24 +01:00
segwit_addr.py file reorganization with top-level module 2018-07-13 14:01:37 +02:00
servers.json Update Johoe's server address (#5363) 2019-05-19 17:53:22 +02:00
servers_regtest.json file reorganization with top-level module 2018-07-13 14:01:37 +02:00
servers_testnet.json servers: update testnet default list 2019-04-27 23:10:12 +02:00
simple_config.py coinchooser: don't spend buckets with negative effective value 2019-06-20 22:42:50 +02:00
storage.py storage: fix some madness about get_data_ref() and put() interacting badly 2019-06-06 19:49:06 +02:00
synchronizer.py interface: follow-up 6cc70bc7a2 2019-05-15 19:56:16 +02:00
transaction.py TxOutput usage: trivial clean-up 2019-06-19 21:59:49 +02:00
util.py update block explorer URL for blockchain.info 2019-06-11 19:19:43 +02:00
verifier.py logging: make console log lines shorter 2019-05-02 15:19:11 +02:00
version.py prepare release 3.3.6 2019-05-16 19:02:20 +02:00
wallet.py coinchooser: don't spend buckets with negative effective value 2019-06-20 22:42:50 +02:00
websockets.py logging: basics 2019-05-02 15:19:03 +02:00
x509.py logging: cli options to filter for modules using -v 2019-05-02 15:19:10 +02:00