Feed environment data into RNG initializers by sipa · Pull Request #17270 · bitcoin/bitcoin

sipa · 2019-10-27T01:26:21Z

This introduces a new randomenv module that queries varies non-cryptographic (and non-RNG) sources of entropy available on the system; things like user IDs, system configuration, time, statistics, CPUID data.

The idea is that these provide a fallback in scenarios where system entropy is somehow broken (note that if system entropy fails we will abort regardless; this is only meant to function as a last resort against undetected failure). It includes some data sources OpenSSL currently uses, and more.

The separation between random and randomenv is a bit arbitrary, but I felt that all this "non-essential" functionality deserved to be separated from the core random module.

DrahtBot · 2019-10-27T02:24:48Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#12557 ([WIP] 64 bit iOS device support by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

src/randomenv.cpp

laanwj · 2019-10-27T10:19:50Z

Errors on MacOSX (looks like it doesn't have environ and CLOCK_MONOTONIC):

andomenv.cpp:177:19: error: use of undeclared identifier 'CLOCK_MONOTONIC'
    clock_gettime(CLOCK_MONOTONIC, &ts);
randomenv.cpp:222:64: error: use of undeclared identifier 'environ'
    hasher << &x << &RandAddStaticEnv << &malloc << &errno << &environ << addr;                                                
randomenv.cpp:282:9: error: use of undeclared identifier 'environ'
    if (environ) {
randomenv.cpp:283:28: error: use of undeclared identifier 'environ'
        for (size_t i = 0; environ[i]; ++i) {
randomenv.cpp:284:48: error: use of undeclared identifier 'environ'; did you mean 'union'?
            hasher.Write((const unsigned char*)environ[i], strlen(environ[i]));
                                               ^~~~~~~
                                               union
randomenv.cpp:284:48: error: expected expression
randomenv.cpp:284:67: error: use of undeclared identifier 'environ'
            hasher.Write((const unsigned char*)environ[i], strlen(environ[i]));
                                                                 ^
7 errors generated.

practicalswift · 2019-10-27T20:30:07Z

Concept ACK: good idea

sipa · 2019-10-27T21:34:22Z

I think I've addressed a compatibility/build issues. Can someone try this on OSX, and perhaps some BSD flavor?

RandyMcMillan · 2019-10-27T22:43:56Z

@sipa,
It runs....
https://github.com/RandyMcMillan/bitcoin/blob/201910_seedrandom/debug.log
https://github.com/RandyMcMillan/bitcoin/blob/201910_seedrandom/mojave10.14.6build.txt

RandyMcMillan · 2019-10-27T22:56:04Z

travis-ci > https://travis-ci.org/RandyMcMillan/bitcoin/jobs/603655433

promag

Concept ACK, nice commit sequence.

fanquake · 2019-10-28T14:13:04Z

Can someone try this on OSX, and perhaps some BSD flavor?

I get the following error when building eed8cfec452a538d644844fb96606e6bc24f47c8 on OpenBSD 6.6:

  CXX      libbitcoin_util_a-randomenv.o
In file included from randomenv.cpp:37:
/usr/include/netinet/ip.h:67:19: error: field has incomplete type 'struct in_addr'
        struct    in_addr ip_src, ip_dst; /* source and dest address */
                          ^
/usr/include/netinet/ip.h:67:11: note: forward declaration of 'in_addr'
        struct    in_addr ip_src, ip_dst; /* source and dest address */
                  ^
/usr/include/netinet/ip.h:67:27: error: field has incomplete type 'struct in_addr'
        struct    in_addr ip_src, ip_dst; /* source and dest address */
                                  ^
/usr/include/netinet/ip.h:67:11: note: forward declaration of 'in_addr'
        struct    in_addr ip_src, ip_dst; /* source and dest address */
                  ^
/usr/include/netinet/ip.h:181:19: error: field has incomplete type 'struct in_addr'
                        struct in_addr ipt_addr;
                                       ^
/usr/include/netinet/ip.h:67:11: note: forward declaration of 'in_addr'
        struct    in_addr ip_src, ip_dst; /* source and dest address */
                  ^
randomenv.cpp:101:57: error: unknown type name 'sockaddr_in'; did you mean 'sockaddr'?
        hasher.Write((const unsigned char*)addr, sizeof(sockaddr_in));
                                                        ^~~~~~~~~~~
                                                        sockaddr
/usr/include/sys/socket.h:207:8: note: 'sockaddr' declared here
struct sockaddr {
       ^
randomenv.cpp:104:57: error: unknown type name 'sockaddr_in6'; did you mean 'sockaddr'?
        hasher.Write((const unsigned char*)addr, sizeof(sockaddr_in6));
                                                        ^~~~~~~~~~~~
                                                        sockaddr
/usr/include/sys/socket.h:207:8: note: 'sockaddr' declared here
struct sockaddr {
       ^
5 errors generated.

laanwj · 2019-10-28T15:37:07Z

Log on FreeBSD 12.0-RELEASE-p10 (same complaint, about sockaddr)

Details

Making all in src
gmake[1]: Entering directory '/usr/home/user/src/bitcoin/src'
gmake[2]: Entering directory '/usr/home/user/src/bitcoin/src'
  CXX      libbitcoin_util_a-randomenv.o
In file included from randomenv.cpp:37:
/usr/include/netinet/ip.h:71:17: error: field has incomplete type 'struct in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                        ^
/usr/include/netinet/ip.h:71:9: note: forward declaration of 'in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                ^
/usr/include/netinet/ip.h:71:24: error: field has incomplete type 'struct in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                               ^
/usr/include/netinet/ip.h:71:9: note: forward declaration of 'in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                ^
/usr/include/netinet/ip.h:188:19: error: field has incomplete type 'struct in_addr'
                        struct in_addr ipt_addr;
                                       ^
/usr/include/netinet/ip.h:71:9: note: forward declaration of 'in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                ^
/usr/include/netinet/ip.h:223:17: error: field has incomplete type 'struct in_addr'
        struct  in_addr ippseudo_src;   /* source internet address */
                        ^
/usr/include/netinet/ip.h:71:9: note: forward declaration of 'in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                ^
/usr/include/netinet/ip.h:224:17: error: field has incomplete type 'struct in_addr'
        struct  in_addr ippseudo_dst;   /* destination internet address */
                        ^
/usr/include/netinet/ip.h:71:9: note: forward declaration of 'in_addr'
        struct  in_addr ip_src,ip_dst;  /* source and dest address */
                ^
randomenv.cpp:101:57: error: unknown type name 'sockaddr_in'; did you mean 'sockaddr'?
        hasher.Write((const unsigned char*)addr, sizeof(sockaddr_in));
                                                        ^~~~~~~~~~~
                                                        sockaddr
/usr/include/sys/socket.h:328:8: note: 'sockaddr' declared here
struct sockaddr {
       ^
randomenv.cpp:104:57: error: unknown type name 'sockaddr_in6'; did you mean 'sockaddr'?
        hasher.Write((const unsigned char*)addr, sizeof(sockaddr_in6));
                                                        ^~~~~~~~~~~~
                                                        sockaddr
/usr/include/sys/socket.h:328:8: note: 'sockaddr' declared here
struct sockaddr {
       ^
7 errors generated.
gmake[2]: *** [Makefile:8866: libbitcoin_util_a-randomenv.o] Error 1
gmake[2]: Leaving directory '/usr/home/user/src/bitcoin/src'
gmake[1]: *** [Makefile:14135: all-recursive] Error 1
gmake[1]: Leaving directory '/usr/home/user/src/bitcoin/src'
gmake: *** [Makefile:774: all-recursive] Error 1

maflcko · 2019-10-28T15:37:52Z

Building freeBSD here: https://cirrus-ci.com/task/5690364417015808

Edit: Same error: https://cirrus-ci.com/task/5690364417015808?command=make#L734

maflcko · 2019-10-28T15:50:40Z

Does the comment in random.h need to be updated?

fanquake · 2019-10-28T15:58:28Z

Does the comment in random.h need to be updated?

If that is updated, you could also cherry-pick 770cd96 out of #17265 which contains some related corrections.

maflcko · 2019-10-28T18:41:44Z

New freeBSD build: https://cirrus-ci.com/task/4712584309112832

f93fc61 Put bounds on the number of CPUID leaves explored (Pieter Wuille) ba2c5fe Fix CPUID subleaf iteration (Pieter Wuille) Pull request description: This fixes bitcoin#17523. The code to determine which CPUID subleaves to explore was incorrect in bitcoin#17270. The new code here is based on Intel's reference documentation for CPUID (a document called "Intel® Processor Identification and the CPUID Instruction - Application Note 485", which I cannot actually find on their own website). ACKs for top commit: laanwj: ACK f93fc61 jonatack: ACK f93fc61 code review, tested rebased on current master bb862d7 with Debian 4.19 x86_64 mzumsande: ACK f93fc61, reviewed code and compared with the intel doc, tested on an AMD and an Intel processor. Tree-SHA512: 2790b326fa397b736c0f39f25807bea57de2752fdd58bf6693d044b8cb26df36c11cce165a334b471f8e33724f10e3b76edab5cc4e0e7776601aabda13277245

…rv64 toolchain issue eafd259 build: Add NX workaround for RV64 (Wladimir J. van der Laan) f6e4225 build: Allow export of environ symbols (Wladimir J. van der Laan) Pull request description: This export was introduced in #17270 which added ``` //! Necessary on some platforms extern char** environ; ``` This should (finally) make the gitian build pass again (fix issue #17525.). Built on top of #17538 which should be merged first. Top commit has no ACKs. Tree-SHA512: 5c2054d52d0957aec3dc945b76d8e219187d22dc03889e7a88fb76049bf8e4a3e9f4da00dd1e9dd0351211f8e70d1a1b8ad7244f0348dab698e9d14b9d0c0bd4

…around rv64 toolchain issue eafd259 build: Add NX workaround for RV64 (Wladimir J. van der Laan) f6e4225 build: Allow export of environ symbols (Wladimir J. van der Laan) Pull request description: This export was introduced in bitcoin#17270 which added ``` //! Necessary on some platforms extern char** environ; ``` This should (finally) make the gitian build pass again (fix issue bitcoin#17525.). Built on top of bitcoin#17538 which should be merged first. Top commit has no ACKs. Tree-SHA512: 5c2054d52d0957aec3dc945b76d8e219187d22dc03889e7a88fb76049bf8e4a3e9f4da00dd1e9dd0351211f8e70d1a1b8ad7244f0348dab698e9d14b9d0c0bd4

55b2cb1 random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake) 461e547 doc: correct random.h docs after #17270 (fanquake) Pull request description: The usage of `MilliSleep()` in SeedPeriodic (previously SeedSleep) was [removed](d61f2bb) in #17270, meaning it, and its users can now be marked `noexcept`. This also corrects the docs in random.h for some of the changes in #17270. ACKs for top commit: practicalswift: ACK 55b2cb1 laanwj: ACK 55b2cb1 sipa: ACK 55b2cb1 Tree-SHA512: 672d369796e7c4f9b4d98dc545e5454999fa1bef373871994a26041d6163c58909e2255e4f820d3ef011679aa3392754eb57477306a89f5fd3d57e2bd7f0811a

… noexcept 55b2cb1 random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake) 461e547 doc: correct random.h docs after bitcoin#17270 (fanquake) Pull request description: The usage of `MilliSleep()` in SeedPeriodic (previously SeedSleep) was [removed](bitcoin@d61f2bb) in bitcoin#17270, meaning it, and its users can now be marked `noexcept`. This also corrects the docs in random.h for some of the changes in bitcoin#17270. ACKs for top commit: practicalswift: ACK 55b2cb1 laanwj: ACK 55b2cb1 sipa: ACK 55b2cb1 Tree-SHA512: 672d369796e7c4f9b4d98dc545e5454999fa1bef373871994a26041d6163c58909e2255e4f820d3ef011679aa3392754eb57477306a89f5fd3d57e2bd7f0811a

This export was introduced in bitcoin#17270 which added ``` //! Necessary on some platforms extern char** environ; ``` Github-Pull: bitcoin#17569 Rebased-From: f6e4225

dc9305b random: don't special case clock usage on macOS (fanquake) Pull request description: `clock_gettime()`, `CLOCK_MONOTONIC` and `CLOCK_REALTIME` are all available for use on macOS (now that we require macOS >=10.12 and build against 10.14). Use them rather than the [deprecated](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Mach/Mach.html) `mach_timespec_t` time API. I mentioned the possibility for this change [in #17270](#17270 (comment)). [master](1dbf335): ```bash 2019-12-23T20:49:43Z Feeding 216 bytes of dynamic environment data into RNG 2019-12-23T20:50:43Z Feeding 216 bytes of dynamic environment data into RNG ``` This PR: ```bash 2019-12-23T20:32:41Z Feeding 232 bytes of dynamic environment data into RNG 2019-12-23T20:33:42Z Feeding 232 bytes of dynamic environment data into RNG ``` ~~Depends on #16392.~~ Merged. ACKs for top commit: laanwj: ACK dc9305b Tree-SHA512: 18c2f336ea628f9cf7339b817381d230a18893fd9c0351bf99a39ca6f45c5b0a20af9d599d48d6c09515627d5edafa91337c17f9f790264251d2cdcb3763bbd5

dc9305b random: don't special case clock usage on macOS (fanquake) Pull request description: `clock_gettime()`, `CLOCK_MONOTONIC` and `CLOCK_REALTIME` are all available for use on macOS (now that we require macOS >=10.12 and build against 10.14). Use them rather than the [deprecated](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Mach/Mach.html) `mach_timespec_t` time API. I mentioned the possibility for this change [in bitcoin#17270](bitcoin#17270 (comment)). [master](1dbf335): ```bash 2019-12-23T20:49:43Z Feeding 216 bytes of dynamic environment data into RNG 2019-12-23T20:50:43Z Feeding 216 bytes of dynamic environment data into RNG ``` This PR: ```bash 2019-12-23T20:32:41Z Feeding 232 bytes of dynamic environment data into RNG 2019-12-23T20:33:42Z Feeding 232 bytes of dynamic environment data into RNG ``` ~~Depends on bitcoin#16392.~~ Merged. ACKs for top commit: laanwj: ACK dc9305b Tree-SHA512: 18c2f336ea628f9cf7339b817381d230a18893fd9c0351bf99a39ca6f45c5b0a20af9d599d48d6c09515627d5edafa91337c17f9f790264251d2cdcb3763bbd5

This export was introduced in bitcoin#17270 which added ``` //! Necessary on some platforms extern char** environ; ```

The usage of MilliSleep() in SeedPeriodic (previously SeedSleep) was removed in bitcoin#17270, meaning it, and its users can now be marked noexcept.

Summary: ``` This should have been part of #17151. ``` Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@b51bae1 Depends on D6157. Test Plan: Read the comments. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6158

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@cea3902 Test Plan: ninja make Run the build-win64 CI build plan. Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D6159

Summary: ``` This sort of data is also used by OpenSSL. ``` Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@c2a262a Depends on D6159. Test Plan: ninja all check-all Run the build-win64 and build-osx CI build plans. Reviewers: #bitcoin_abc, jasonbcox Reviewed By: #bitcoin_abc, jasonbcox Differential Revision: https://reviews.bitcoinabc.org/D6160

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@723c796 Test Plan: ninja all check-all Run the Gitian builds. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Subscribers: deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6161

Summary: ``` This based on code by Gregory Maxwell. ``` Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@2554c1b Depends on D6161. Test Plan: ninja all check-all Run the Gitian builds (will run symbol-check). Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6162

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@a81c494 Test Plan: ninja all check-all Build for OSX (native and cross build), check the binary runs. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Subscribers: deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6201

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@11793ea Test Plan: ninja all check-all Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6205

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@483b942 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6206

Summary: ``` * Instead of calling RandAddSeedSleep anytime the scheduler goes idle, call its replacement (RandAddSeedPeriodic) just once per minute. This has better guarantees of actually being run, and helps limit how frequently the dynamic env data is gathered. * Since this code runs once per minute regardless now, we no longer need to keep track of the last time strengthening was run; just do it always. * Make strengthening time context dependent (100 ms at startup, 10 ms once per minute afterwards). ``` Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@d61f2bb Test Plan: ninja all check-all Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6207

Summary: ``` Also switch to chrono based types. ``` Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@64e1e02 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6208

Summary: Completes backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@d1c0277 Depends on D6207. Test Plan: ninja all check-all ./src/bitcoind Check in the log that the amount of environment data feeded into RNG is logged. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6209

Summary: Partial backport of core [[bitcoin/bitcoin#17270 | PR17270]]: bitcoin/bitcoin@723c796 Test Plan: ninja all check-all Run the Gitian builds. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Subscribers: deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6161

hebasto · 2022-08-16T07:01:29Z

src/randomenv.cpp

+//! Necessary on some platforms
+extern char** environ;


@sipa
While compiling natively on Windows with MSVC, receiving C4273 warnings:

randomenv.cpp C:\Users\hebasto\bitcoin\src\randomenv.cpp(60,22): warning C4273: '__p__environ': inconsistent dll linkage [C:\Users\hebasto\bitcoin\build\src\util\bitcoin_util.vcxproj] C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\stdlib.h(1158,29): message : see previous definition of '__p__environ' [C:\Users\hebasto\bitcoin\build\src\util\bitcoin_util. vcxproj]

randomenv.cpp C:\Users\hebasto\bitcoin\src\randomenv.cpp(60,22): warning C4273: '__p__environ': inconsistent dll linkage [C:\Users\hebasto\bitcoin\build\src\kernel\bitcoinkernel.vcxproj] C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\stdlib.h(1158,29): message : see previous definition of '__p__environ' [C:\Users\hebasto\bitcoin\build\src\kernel\bitcoinkern el.vcxproj]

Considering https://docs.microsoft.com/en-us/cpp/c-runtime-library/environ-wenviron, is this code correct for MSVC?

nm, no issues with the latest build.

DrahtBot added the Build system label Oct 27, 2019

meshcollider added Utils/log/libs and removed Build system labels Oct 27, 2019

sipa force-pushed the 201910_seedrandom branch 4 times, most recently from 0971ae9 to 0c6d5cb Compare October 27, 2019 02:01

laanwj reviewed Oct 27, 2019

View reviewed changes

src/randomenv.cpp Outdated Show resolved Hide resolved

laanwj mentioned this pull request Oct 27, 2019

Remove OpenSSL #17265

Merged

sipa force-pushed the 201910_seedrandom branch 2 times, most recently from c197258 to 4b12381 Compare October 27, 2019 20:28

sipa force-pushed the 201910_seedrandom branch 2 times, most recently from 0b1d233 to eed8cfe Compare October 27, 2019 20:51

promag reviewed Oct 28, 2019

View reviewed changes

sipa force-pushed the 201910_seedrandom branch 4 times, most recently from e604e52 to 8470cbb Compare October 28, 2019 18:40

Sjors mentioned this pull request Nov 28, 2019

Suppress false positive warning about uninitialized entropy buffers #17627

Closed

fanquake mentioned this pull request Dec 24, 2019

random: don't special case clock usage on macOS #17800

Merged

Naviabheeman mentioned this pull request Apr 3, 2020

Remove Openssl dependency from Tapyrus chaintope/tapyrus-core#61

Closed

MarkLTZ mentioned this pull request Apr 4, 2020

Bitcoin PR tracking litecoinz-core/litecoinz#14

Closed

HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this pull request Apr 17, 2020

build: Allow export of environ symbols

7f9303b

This export was introduced in bitcoin#17270 which added ``` //! Necessary on some platforms extern char** environ; ```

HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this pull request Apr 17, 2020

doc: correct random.h docs after bitcoin#17270

ece4c49

Fuzzbawls mentioned this pull request Mar 31, 2021

[Refactor] Update RNG code from upstream PIVX-Project/PIVX#2278

Merged

hebasto reviewed Aug 16, 2022

View reviewed changes

Conversation

sipa commented Oct 27, 2019

Uh oh!

DrahtBot commented Oct 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Conflicts

Uh oh!

Uh oh!

laanwj commented Oct 27, 2019

Uh oh!

practicalswift commented Oct 27, 2019

Uh oh!

sipa commented Oct 27, 2019

Uh oh!

RandyMcMillan commented Oct 27, 2019

Uh oh!

RandyMcMillan commented Oct 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

promag left a comment

Choose a reason for hiding this comment

Uh oh!

fanquake commented Oct 28, 2019

Uh oh!

laanwj commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maflcko commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maflcko commented Oct 28, 2019

Uh oh!

fanquake commented Oct 28, 2019

Uh oh!

maflcko commented Oct 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto Aug 16, 2022

Choose a reason for hiding this comment

Uh oh!

hebasto Oct 24, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

16 participants

DrahtBot commented Oct 27, 2019 •

edited

Loading

RandyMcMillan commented Oct 27, 2019 •

edited

Loading

laanwj commented Oct 28, 2019 •

edited

Loading

maflcko commented Oct 28, 2019 •

edited

Loading

maflcko commented Oct 28, 2019 •

edited

Loading