Skip to content

Commit d685554

Browse files
author
Leonardo Alt
committed
Create option for metadata hash
1 parent c72d1ff commit d685554

22 files changed

Lines changed: 273 additions & 118 deletions

‎Changelog.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@ Breaking changes:
44
* ABI: remove the deprecated ``constant`` and ``payable`` fields.
55
* ABI: the ``type`` field is now required and no longer specified to default to ``function``.
66
* Commandline interface: remove the text-based ast printer (``--ast``).
7+
* Command line interface: Switch to the new error reporter by default. ``--old-reporter`` falls back to the deprecated old error reporter.
8+
* Command line interface: Add option to disable or choose hash method between IPFS and Swarm for the bytecode metadata.
79
* General: Disallow explicit conversions from external function types to ``address`` and add a member called ``address`` to them as replacement.
810
* General: New reserved keywords: ``virtual``.
11+
* Standard JSON Interface: Add option to disable or choose hash method between IPFS and Swarm for the bytecode metadata.
912
* Type checker: Resulting type of exponentiation is equal to the type of the base. Also allow signed types for the base.
10-
* Command line interface: Switch to the new error reporter by default. ``--old-reporter`` falls back to the deprecated old error reporter.
1113

1214

1315
Language Features:

‎docs/metadata.rst‎

Lines changed: 29 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,18 @@ this file to query the compiler version, the sources used, the ABI and NatSpec
1010
documentation to more safely interact with the contract and verify its source
1111
code.
1212

13-
The compiler appends a Swarm hash of the metadata file to the end of the
14-
bytecode (for details, see below) of each contract, so that you can retrieve
15-
the file in an authenticated way without having to resort to a centralized
16-
data provider.
17-
18-
You have to publish the metadata file to Swarm (or another service) so that
19-
others can access it. You create the file by using the ``solc --metadata``
13+
The compiler appends by default the IPFS hash of the metadata file to the end
14+
of the bytecode (for details, see below) of each contract, so that you can
15+
retrieve the file in an authenticated way without having to resort to a
16+
centralized data provider. The other available options are the Swarm hash and
17+
not appending the metadata hash to the bytecode. These can be configured via
18+
the :ref:`Standard JSON Interface<compiler-api>`.
19+
20+
You have to publish the metadata file to IPFS, Swarm, or another service so
21+
that others can access it. You create the file by using the ``solc --metadata``
2022
command that generates a file called ``ContractName_meta.json``. It contains
21-
Swarm references to the source code, so you have to upload all source files and
22-
the metadata file.
23+
IPFS and Swarm references to the source code, so you have to upload all source
24+
files and the metadata file.
2325

2426
The metadata file has the following format. The example below is presented in a
2527
human-readable way. Properly formatted metadata should use quotes correctly,
@@ -86,7 +88,9 @@ explanatory purposes.
8688
},
8789
metadata: {
8890
// Reflects the setting used in the input json, defaults to false
89-
useLiteralContent: true
91+
useLiteralContent: true,
92+
// Reflects the setting used in the input json, defaults to "ipfs"
93+
bytecodeHash: "ipfs"
9094
}
9195
// Required for Solidity: File and name of the contract or library this
9296
// metadata is created for.
@@ -111,8 +115,8 @@ explanatory purposes.
111115
}
112116
113117
.. warning::
114-
Since the bytecode of the resulting contract contains the metadata hash, any
115-
change to the metadata results in a change of the bytecode. This includes
118+
Since the bytecode of the resulting contract contains the metadata hash by default, any
119+
change to the metadata might result in a change of the bytecode. This includes
116120
changes to a filename or path, and since the metadata includes a hash of all the
117121
sources used, a single whitespace change results in different metadata, and
118122
different bytecode.
@@ -124,46 +128,46 @@ Encoding of the Metadata Hash in the Bytecode
124128
=============================================
125129

126130
Because we might support other ways to retrieve the metadata file in the future,
127-
the mapping ``{"bzzr1": <Swarm hash>, "solc": <compiler version>}`` is stored
131+
the mapping ``{"ipfs": <IPFS hash>, "solc": <compiler version>}`` is stored
128132
`CBOR <https://tools.ietf.org/html/rfc7049>`_-encoded. Since the mapping might
129133
contain more keys (see below) and the beginning of that
130134
encoding is not easy to find, its length is added in a two-byte big-endian
131135
encoding. The current version of the Solidity compiler usually adds the following
132136
to the end of the deployed bytecode::
133137

134138
0xa2
135-
0x65 'b' 'z' 'z' 'r' '1' 0x58 0x20 <32 bytes swarm hash>
139+
0x64 'i' 'p' 'f' 's' 0x58 0x22 <34 bytes IPFS hash>
136140
0x64 's' 'o' 'l' 'c' 0x43 <3 byte version encoding>
137141
0x00 0x32
138142

139143
So in order to retrieve the data, the end of the deployed bytecode can be checked
140-
to match that pattern and use the Swarm hash to retrieve the file.
144+
to match that pattern and use the IPFS hash to retrieve the file.
141145

142146
Whereas release builds of solc use a 3 byte encoding of the version as shown
143147
above (one byte each for major, minor and patch version number), prerelease builds
144148
will instead use a complete version string including commit hash and build date.
145149

146150
.. note::
147151
The CBOR mapping can also contain other keys, so it is better to fully
148-
decode the data instead of relying on it starting with ``0xa265``.
152+
decode the data instead of relying on it starting with ``0xa264``.
149153
For example, if any experimental features that affect code generation
150154
are used, the mapping will also contain ``"experimental": true``.
151155

152156
.. note::
153-
The compiler currently uses the "swarm version 1" hash of the metadata,
154-
but this might change in the future, so do not rely on this sequence
155-
to start with ``0xa2 0x65 'b' 'z' 'z' 'r' '1'``. We might also
156-
add additional data to this CBOR structure, so the
157-
best option is to use a proper CBOR parser.
157+
The compiler currently uses the IPFS hash of the metadata by default, but
158+
it may also use the bzzr1 hash or some other hash in the future, so do
159+
not rely on this sequence to start with ``0xa2 0x64 'i' 'p' 'f' 's'``. We
160+
might also add additional data to this CBOR structure, so the best option
161+
is to use a proper CBOR parser.
158162

159163

160164
Usage for Automatic Interface Generation and NatSpec
161165
====================================================
162166

163167
The metadata is used in the following way: A component that wants to interact
164-
with a contract (e.g. Mist or any wallet) retrieves the code of the contract, from that
165-
the Swarm hash of a file which is then retrieved.
166-
That file is JSON-decoded into a structure like above.
168+
with a contract (e.g. Mist or any wallet) retrieves the code of the contract,
169+
from that the IPFS/Swarm hash of a file which is then retrieved. That file
170+
is JSON-decoded into a structure like above.
167171

168172
The component can then use the ABI to automatically generate a rudimentary
169173
user interface for the contract.
@@ -177,7 +181,7 @@ For additional information, read :doc:`Ethereum Natural Language Specification (
177181
Usage for Source Code Verification
178182
==================================
179183

180-
In order to verify the compilation, sources can be retrieved from Swarm
184+
In order to verify the compilation, sources can be retrieved from IPFS/Swarm
181185
via the link in the metadata file.
182186
The compiler of the correct version (which is checked to be part of the "official" compilers)
183187
is invoked on that input with the specified settings. The resulting

‎docs/using-the-compiler.rst‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,12 @@ Input Description
236236
// Metadata settings (optional)
237237
"metadata": {
238238
// Use only literal content and not URLs (false by default)
239-
"useLiteralContent": true
239+
"useLiteralContent": true,
240+
// Use the given hash method for the metadata hash that is appended to the bytecode.
241+
// The metadata hash can be removed from the bytecode via option "none".
242+
// The other options are "ipfs" and "bzzr1".
243+
// If the option is omitted, "ipfs" is used by default.
244+
"bytecodeHash": "ipfs"
240245
},
241246
// Addresses of the libraries. If not all libraries are given here,
242247
// it can result in unlinked objects whose output data is different.

‎libsolidity/interface/CompilerStack.cpp‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,13 @@ void CompilerStack::useMetadataLiteralSources(bool _metadataLiteralSources)
159159
m_metadataLiteralSources = _metadataLiteralSources;
160160
}
161161

162+
void CompilerStack::setMetadataHash(MetadataHash _metadataHash)
163+
{
164+
if (m_stackState >= ParsingPerformed)
165+
BOOST_THROW_EXCEPTION(CompilerError() << errinfo_comment("Must set metadata hash before parsing."));
166+
m_metadataHash = _metadataHash;
167+
}
168+
162169
void CompilerStack::addSMTLib2Response(h256 const& _hash, string const& _response)
163170
{
164171
if (m_stackState >= ParsingPerformed)
@@ -182,6 +189,7 @@ void CompilerStack::reset(bool _keepSettings)
182189
m_generateEWasm = false;
183190
m_optimiserSettings = OptimiserSettings::minimal();
184191
m_metadataLiteralSources = false;
192+
m_metadataHash = MetadataHash::IPFS;
185193
}
186194
m_globalContext.reset();
187195
m_scopes.clear();
@@ -1155,6 +1163,10 @@ string CompilerStack::createMetadata(Contract const& _contract) const
11551163

11561164
if (m_metadataLiteralSources)
11571165
meta["settings"]["metadata"]["useLiteralContent"] = true;
1166+
1167+
static vector<string> hashes{"ipfs", "bzzr1", "none"};
1168+
meta["settings"]["metadata"]["bytecodeHash"] = hashes.at(unsigned(m_metadataHash));
1169+
11581170
meta["settings"]["evmVersion"] = m_evmVersion.name();
11591171
meta["settings"]["compilationTarget"][_contract.contract->sourceUnitName()] =
11601172
_contract.contract->annotation().canonicalName;
@@ -1263,7 +1275,17 @@ class MetadataCBOREncoder
12631275
bytes CompilerStack::createCBORMetadata(string const& _metadata, bool _experimentalMode)
12641276
{
12651277
MetadataCBOREncoder encoder;
1266-
encoder.pushBytes("bzzr1", dev::bzzr1Hash(_metadata).asBytes());
1278+
1279+
if (m_metadataHash == MetadataHash::IPFS)
1280+
{
1281+
solAssert(_metadata.length() < 1024 * 256, "Metadata too large.");
1282+
encoder.pushBytes("ipfs", dev::ipfsHash(_metadata));
1283+
}
1284+
else if (m_metadataHash == MetadataHash::Bzzr1)
1285+
encoder.pushBytes("bzzr1", dev::bzzr1Hash(_metadata).asBytes());
1286+
else
1287+
solAssert(m_metadataHash == MetadataHash::None, "Invalid metadata hash");
1288+
12671289
if (_experimentalMode)
12681290
encoder.pushBool("experimental", true);
12691291
if (m_release)

‎libsolidity/interface/CompilerStack.h‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,12 @@ class CompilerStack: boost::noncopyable
9292
CompilationSuccessful
9393
};
9494

95+
enum class MetadataHash {
96+
IPFS,
97+
Bzzr1,
98+
None
99+
};
100+
95101
struct Remapping
96102
{
97103
std::string context;
@@ -171,6 +177,11 @@ class CompilerStack: boost::noncopyable
171177
/// Must be set before parsing.
172178
void useMetadataLiteralSources(bool _metadataLiteralSources);
173179

180+
/// Sets whether and which hash should be used
181+
/// to store the metadata in the bytecode.
182+
/// @param _metadataHash can be IPFS, Bzzr1, None
183+
void setMetadataHash(MetadataHash _metadataHash);
184+
174185
/// Sets the sources. Must be set before parsing.
175186
void setSources(StringMap _sources);
176187

@@ -418,6 +429,7 @@ class CompilerStack: boost::noncopyable
418429
langutil::ErrorList m_errorList;
419430
langutil::ErrorReporter m_errorReporter;
420431
bool m_metadataLiteralSources = false;
432+
MetadataHash m_metadataHash = MetadataHash::IPFS;
421433
bool m_parserErrorRecovery = false;
422434
State m_stackState = Empty;
423435
/// Whether or not there has been an error during processing.

‎libsolidity/interface/StandardCompiler.cpp‎

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -378,9 +378,16 @@ boost::optional<Json::Value> checkOptimizerDetail(Json::Value const& _details, s
378378

379379
boost::optional<Json::Value> checkMetadataKeys(Json::Value const& _input)
380380
{
381-
if (_input.isObject() && _input.isMember("useLiteralContent") && !_input["useLiteralContent"].isBool())
382-
return formatFatalError("JSONError", "\"settings.metadata.useLiteralContent\" must be Boolean");
383-
static set<string> keys{"useLiteralContent"};
381+
if (_input.isObject())
382+
{
383+
if (_input.isMember("useLiteralContent") && !_input["useLiteralContent"].isBool())
384+
return formatFatalError("JSONError", "\"settings.metadata.useLiteralContent\" must be Boolean");
385+
386+
static set<string> hashes{"ipfs", "bzzr1", "none"};
387+
if (_input.isMember("bytecodeHash") && !hashes.count(_input["bytecodeHash"].asString()))
388+
return formatFatalError("JSONError", "\"settings.metadata.bytecodeHash\" must be \"ipfs\", \"bzzr1\" or \"none\"");
389+
}
390+
static set<string> keys{"useLiteralContent", "bytecodeHash"};
384391
return checkKeys(_input, keys, "settings.metadata");
385392
}
386393

@@ -710,6 +717,16 @@ boost::variant<StandardCompiler::InputsAndSettings, Json::Value> StandardCompile
710717
return *result;
711718

712719
ret.metadataLiteralSources = metadataSettings.get("useLiteralContent", Json::Value(false)).asBool();
720+
if (metadataSettings.isMember("bytecodeHash"))
721+
{
722+
auto metadataHash = metadataSettings["bytecodeHash"].asString();
723+
ret.metadataHash =
724+
metadataHash == "ipfs" ?
725+
CompilerStack::MetadataHash::IPFS :
726+
metadataHash == "bzzr1" ?
727+
CompilerStack::MetadataHash::Bzzr1 :
728+
CompilerStack::MetadataHash::None;
729+
}
713730

714731
Json::Value outputSelection = settings.get("outputSelection", Json::Value());
715732

@@ -735,6 +752,7 @@ Json::Value StandardCompiler::compileSolidity(StandardCompiler::InputsAndSetting
735752
compilerStack.setOptimiserSettings(std::move(_inputsAndSettings.optimiserSettings));
736753
compilerStack.setLibraries(_inputsAndSettings.libraries);
737754
compilerStack.useMetadataLiteralSources(_inputsAndSettings.metadataLiteralSources);
755+
compilerStack.setMetadataHash(_inputsAndSettings.metadataHash);
738756
compilerStack.setRequestedContractNames(requestedContractNames(_inputsAndSettings.outputSelection));
739757

740758
compilerStack.enableIRGeneration(isIRRequested(_inputsAndSettings.outputSelection));

‎libsolidity/interface/StandardCompiler.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,7 @@ class StandardCompiler: boost::noncopyable
6868
OptimiserSettings optimiserSettings = OptimiserSettings::minimal();
6969
std::map<std::string, h160> libraries;
7070
bool metadataLiteralSources = false;
71+
CompilerStack::MetadataHash metadataHash = CompilerStack::MetadataHash::IPFS;
7172
Json::Value outputSelection;
7273
};
7374

‎solc/CommandLineInterface.cpp‎

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,15 +122,18 @@ static string const g_strInputFile = "input-file";
122122
static string const g_strInterface = "interface";
123123
static string const g_strYul = "yul";
124124
static string const g_strIR = "ir";
125+
static string const g_strIPFS = "ipfs";
125126
static string const g_strEWasm = "ewasm";
126127
static string const g_strLicense = "license";
127128
static string const g_strLibraries = "libraries";
128129
static string const g_strLink = "link";
129130
static string const g_strMachine = "machine";
130131
static string const g_strMetadata = "metadata";
132+
static string const g_strMetadataHash = "metadata-hash";
131133
static string const g_strMetadataLiteral = "metadata-literal";
132134
static string const g_strNatspecDev = "devdoc";
133135
static string const g_strNatspecUser = "userdoc";
136+
static string const g_strNone = "none";
134137
static string const g_strOpcodes = "opcodes";
135138
static string const g_strOptimize = "optimize";
136139
static string const g_strOptimizeRuns = "optimize-runs";
@@ -144,6 +147,7 @@ static string const g_strSrcMap = "srcmap";
144147
static string const g_strSrcMapRuntime = "srcmap-runtime";
145148
static string const g_strStandardJSON = "standard-json";
146149
static string const g_strStrictAssembly = "strict-assembly";
150+
static string const g_strSwarm = "swarm";
147151
static string const g_strPrettyJson = "pretty-json";
148152
static string const g_strVersion = "version";
149153
static string const g_strIgnoreMissingFiles = "ignore-missing";
@@ -174,6 +178,7 @@ static string const g_argLibraries = g_strLibraries;
174178
static string const g_argLink = g_strLink;
175179
static string const g_argMachine = g_strMachine;
176180
static string const g_argMetadata = g_strMetadata;
181+
static string const g_argMetadataHash = g_strMetadataHash;
177182
static string const g_argMetadataLiteral = g_strMetadataLiteral;
178183
static string const g_argNatspecDev = g_strNatspecDev;
179184
static string const g_argNatspecUser = g_strNatspecUser;
@@ -218,6 +223,14 @@ static set<string> const g_machineArgs
218223
g_streWasm
219224
};
220225

226+
/// Possible arguments to for --metadata-hash
227+
static set<string> const g_metadataHashArgs
228+
{
229+
g_strIPFS,
230+
g_strSwarm,
231+
g_strNone
232+
};
233+
221234
static void version()
222235
{
223236
sout() <<
@@ -696,7 +709,12 @@ Allowed options)",
696709
"Switch to linker mode, ignoring all options apart from --libraries "
697710
"and modify binaries in place."
698711
)
699-
(g_argMetadataLiteral.c_str(), "Store referenced sources are literal data in the metadata output.")
712+
(
713+
g_argMetadataHash.c_str(),
714+
po::value<string>()->value_name(boost::join(g_metadataHashArgs, ",")),
715+
"Choose hash method for the bytecode metadata or disable it."
716+
)
717+
(g_argMetadataLiteral.c_str(), "Store referenced sources as literal data in the metadata output.")
700718
(
701719
g_argAllowPaths.c_str(),
702720
po::value<string>()->value_name("path(s)"),
@@ -923,6 +941,22 @@ bool CommandLineInterface::processInput()
923941
return link();
924942
}
925943

944+
if (m_args.count(g_argMetadataHash))
945+
{
946+
string hashStr = m_args[g_argMetadataHash].as<string>();
947+
if (hashStr == g_strIPFS)
948+
m_metadataHash = CompilerStack::MetadataHash::IPFS;
949+
else if (hashStr == g_strSwarm)
950+
m_metadataHash = CompilerStack::MetadataHash::Bzzr1;
951+
else if (hashStr == g_strNone)
952+
m_metadataHash = CompilerStack::MetadataHash::None;
953+
else
954+
{
955+
serr() << "Invalid option for --metadata-hash: " << hashStr << endl;
956+
return false;
957+
}
958+
}
959+
926960
m_compiler.reset(new CompilerStack(fileReader));
927961

928962
unique_ptr<SourceReferenceFormatter> formatter;
@@ -935,6 +969,8 @@ bool CommandLineInterface::processInput()
935969
{
936970
if (m_args.count(g_argMetadataLiteral) > 0)
937971
m_compiler->useMetadataLiteralSources(true);
972+
if (m_args.count(g_argMetadataHash))
973+
m_compiler->setMetadataHash(m_metadataHash);
938974
if (m_args.count(g_argInputFile))
939975
m_compiler->setRemappings(m_remappings);
940976
m_compiler->setSources(m_sourceCodes);

‎solc/CommandLineInterface.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,8 @@ class CommandLineInterface
111111
std::unique_ptr<dev::solidity::CompilerStack> m_compiler;
112112
/// EVM version to use
113113
langutil::EVMVersion m_evmVersion;
114+
/// Chosen hash method for the bytecode metadata.
115+
CompilerStack::MetadataHash m_metadataHash = CompilerStack::MetadataHash::IPFS;
114116
/// Whether or not to colorize diagnostics output.
115117
bool m_coloredOutput = true;
116118
};

0 commit comments

Comments
 (0)