Skip to content

wallet: derivehdkey RPC to get xpub at arbitrary path - #32784

Draft
Sjors wants to merge 12 commits into
bitcoin:masterfrom
Sjors:2025/06/gethdkey
Draft

wallet: derivehdkey RPC to get xpub at arbitrary path#32784
Sjors wants to merge 12 commits into
bitcoin:masterfrom
Sjors:2025/06/gethdkey

Conversation

@Sjors

@Sjors Sjors commented Jun 20, 2025

Copy link
Copy Markdown
Member

Adds a derivehdkey RPC that returns an xpub, or optionally the xprv, at an arbitrary BIP32 path (with at least one hardened step), derived from a wallet HD key.

The main use case is coordinating a multisig setup, where each participant shares an xpub derived at a hardened path (e.g. m/87h/0h/0h) distinct from their default single-signature descriptors. See the (updated) doc/multisig-tutorial.md and (updated) functional test to see how that workflow improves.

The first commits are some helpful helpers:

  • key: add DeriveExtKey() helper - performs the actual derivation
  • test: move parse_hd_keypath test to bip32_tests - from psbt_wallet_tests
  • Have ParseHDKeypath handle h derivation marker
  • util: reject out-of-range BIP32 keypath indices - ParseHDKeypath would previously map overflowing values without h to hardened.
  • fuzz: check ParseHDKeypath/WriteHDKeypath round-trip
  • rpc: ParsePathBIP32 helper
  • refactor: add hardened derivation helper - HasHardenedDerivation(), to enforce the "at least one hardened step" rule
  • wallet: generalize GetActiveHDPubKeys helper - extracts code from gethdkeys which derivehdkey needs
  • wallet: add GetExtKey helper - reconstruct an xprv from a wallet xpub (analog of GetKey()); behavior-preserving prep, also simplifies gethdkeys.

Meat and potatoes:

  • rpc: add derivehdkey - the RPC itself, plus the UnusedKey filter on GetHDPubKeys that drives key selection.
  • test: use derivehdkey in M-of-N multisig demo - rewrites the functional multisig test to use the RPC and <0;1> syntax.
  • doc: use derivehdkey in multisig tutorial - same for the prose tutorial.

@DrahtBot DrahtBot changed the title wallet: derivehdkey RPC to get xpub at arbitrary path wallet: derivehdkey RPC to get xpub at arbitrary path Jun 20, 2025
@DrahtBot

DrahtBot commented Jun 20, 2025

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32784.

Reviews

See the guideline and AI policy for information on the review process.

Type Reviewers
Concept ACK rkrux, pseudoramdom

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #35747 (wallet: Fix FillPSBT failing to sign owned inputs when UTXOs disagree by nervana21)
  • #35170 (test: Better test coverage for legacy ParseHDKeypath() by optout21)
  • #35069 (Refactor keypath parser by pythcoiner)
  • #34502 (wallet: remove most asserts of WALLET_FLAG_DESCRIPTORS flag by rkrux)
  • #32861 (Have createwalletdescriptor auto-detect an unused(KEY) by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

  • # Get the activate wpkh() receive descriptor -> # Get the active wpkh() receive descriptor [“activate” is the wrong word here; “active” is the intended meaning.]

Possible places where comparison-specific test macros should replace generic comparisons:

  • [test/functional/wallet_derivehdkey.py] assert xpub_info_1["xpub"] != xpub_info_2["xpub"] -> replace with assert_not_equal(xpub_info_1["xpub"], xpub_info_2["xpub"])

2026-08-05 08:25:34

@DrahtBot

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task lint: https://github.com/bitcoin/bitcoin/runs/44475268764
LLM reason (✨ experimental): The CI failure is caused by errors from the lint check 'py_lint', specifically due to unused imports flagged by ruff.

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

Comment thread src/rpc/util.h Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In util.h: “@params[in] path” → “@param[in] path” [Doxygen tag typo]

Comment thread doc/multisig-tutorial.md Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressesd -> addresses [extra “d” makes “addresses” misspelled]

@rkrux

rkrux commented Jul 18, 2025

Copy link
Copy Markdown
Contributor

Very nice, Concept ACK.
I will review this PR.

@Sjors
Sjors force-pushed the 2025/06/gethdkey branch 2 times, most recently from 380a57f to 017fb68 Compare August 1, 2025 10:52
@Sjors
Sjors force-pushed the 2025/06/gethdkey branch from de552d1 to 3ebd59e Compare July 3, 2026 11:48
@Sjors

Sjors commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Rebased after #32489. I moved the new fuzz: check ParseHDKeypath/WriteHDKeypath round-trip closer to the two ParseHDKeypath commits in the stack.

Sjors added a commit to Sjors/bitcoin that referenced this pull request Jul 3, 2026
…tor' into HEAD

Resolve the createwalletdescriptor conflict between bitcoin#32784 and bitcoin#32861 by keeping bitcoin#32861's fallback to unused KEY descriptors, but adapting the active descriptor lookup to bitcoin#32784's GetHDPubKeys(HDKeyFilter::Active) API and extracting the xpub from the map key.
@sedited
sedited requested review from pseudoramdom, rkrux and w0xlt July 24, 2026 07:38

@pseudoramdom pseudoramdom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since #35606 has now merged, we can simplify the fingerprint handling using the new type-safe helpers.

Comment thread src/key.cpp Outdated
CExtKey descendant = ext_key;
KeyOriginInfo origin;
const CKeyID id = ext_key.key.GetPubKey().GetID();
std::copy(id.begin(), id.begin() + sizeof(origin.fingerprint), origin.fingerprint);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In key: add DeriveExtKey() helper fdfb6ca

Since #35606 is merged and changes fingerprints to the fixed-size KeyFingerprint type, this can become

origin.fingerprint = ext_key.id_key_fingerprint();

Comment thread src/test/bip32_tests.cpp Outdated

KeyOriginInfo expected_origin;
const CKeyID id{master.key.GetPubKey().GetID()};
std::copy(id.begin(), id.begin() + sizeof(expected_origin.fingerprint), expected_origin.fingerprint);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto. We can use

expected_origin.fingerprint = master.id_key_fingerprint();

Comment thread src/test/bip32_tests.cpp Outdated
#include <test/util/setup_common.h>
#include <util/strencodings.h>

#include <algorithm>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be dropped if we use the id_key_fingerprint() suggestion

Comment thread src/wallet/rpc/wallet.cpp Outdated

UniValue res{UniValue::VOBJ};

std::string fingerprint{HexStr(std::span<unsigned char>(child->second.fingerprint, child->second.fingerprint + 4))};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In rpc: add derivehdkey (97e1757)

With the fingerprint changes from #35606 , we can simplify this to

const std::string fingerprint{HexStr(child->second.fingerprint)};

Sjors and others added 12 commits August 5, 2026 10:05
Co-authored-by: w0xlt <94266259+w0xlt@users.noreply.github.com>
ParseHDKeypath() lives in util/bip32, so its unit test belongs in
bip32_tests rather than psbt_wallet_tests. Pure move, no changes to the
test itself; subsequent commits extend it in its new home.
ParseHDKeypath() parsed each path element with ToIntegral<uint32_t>, so
a bare decimal >= 2^31 (e.g. "m/2147483648" == 0x80000000) was silently
treated as "m/0h".

This commit rejects such overflow instead.
GetHDPubKeys() centralizes the descriptor xpub lookup used by gethdkeys and
createwalletdescriptor, and by the derivehdkey RPC added in a later commit.
The HDKeyFilter argument serves gethdkeys' active_only mode (Active vs All)
and createwalletdescriptor's active descriptor selection.

No behavior change, except the dynamic_cast now uses Assert() instead of
gethdkeys' CHECK_NONFATAL, since it is not a recoverable input.
Reconstruct a descriptor's extended private key from its xpub by looking
up the corresponding private key. This is the extended-key analog of
GetKey().

It is used here to simplify gethdkeys, and again by the derivehdkey RPC
introduced in a later commit.
Add an UnusedKey filter to GetHDPubKeys() so the new RPC can prefer
unused(KEY) descriptors before falling back to active descriptors.

Co-authored-by: w0xlt <94266259+w0xlt@users.noreply.github.com>
Use derivehdkey instead of extracting each participant xpub (and
derivation info) from  the listdescriptors output.

Additionally use the new <0;1> descriptor syntax.

Finally this commits adds a few debug log lines, and expand the
explanation for why we use m/44h/1h/0h.
Use derivehdkey instead of extracting each participant xpub
from  the listdescriptors output.

Additionally use the new <0;1> descriptor syntax.
@Sjors
Sjors force-pushed the 2025/06/gethdkey branch from 3ebd59e to 5ff96ba Compare August 5, 2026 08:25
@Sjors

Sjors commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

Rebased and switched to #35606 KeyFingerprint, as suggested by @pseudoramdom.

@DrahtBot

DrahtBot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

🚧 At least one of the CI tasks failed.
Task 32 bit ARM: https://github.com/bitcoin/bitcoin/actions/runs/30988942936/job/92250208138
LLM reason (✨ experimental): CI failed due to a compiler error treated as fatal (-Werror=stringop-overread) in std::array’s memcmp call while building test_bitcoin (psbt_wallet_tests.cpp.o).

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

  • Possibly due to a silent merge conflict (the changes in this pull request being
    incompatible with the current code in the target branch). If so, make sure to rebase on the latest
    commit of the target branch.

  • A sanitizer issue, which can only be found by compiling with the sanitizer and running the
    affected test.

  • An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

@maflcko

maflcko commented Aug 5, 2026

Copy link
Copy Markdown
Member

The ci failure is unrelated, you can just add wno-error=... to the ci config.

@Sjors

Sjors commented Aug 5, 2026

Copy link
Copy Markdown
Member Author

I think I'll just wait for #35896 to land (tested locally that it helps).

@Sjors
Sjors marked this pull request as draft August 5, 2026 16:50
Comment thread src/wallet/rpc/wallet.cpp
if (std::optional<CKey> key = priv ? desc_spkm->GetKey(xpub.pubkey.GetID()) : std::nullopt) {
wallet_xprvs[xpub] = CExtKey(xpub, *key);
wallet_xpubs[xpub].emplace(desc_str, wallet->IsActiveScriptPubKeyMan(*desc_spkm), desc_spkm->HasPrivKey(xpub.pubkey.GetID()));
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we look up the private key directly from each associated desc_spkm inside the existing loop in RPCMethod gethdkeys()?

diff --git a/src/wallet/rpc/wallet.cpp b/src/wallet/rpc/wallet.cpp
index cfba900d22..aa9dae8eb7 100644
--- a/src/wallet/rpc/wallet.cpp
+++ b/src/wallet/rpc/wallet.cpp
@@ -704,10 +704,8 @@ RPCMethod gethdkeys()
                     bool ok = desc_spkm->GetDescriptorString(desc_str, /*priv=*/false);
                     CHECK_NONFATAL(ok);
                     wallet_xpubs[xpub].emplace(desc_str, wallet->IsActiveScriptPubKeyMan(*desc_spkm), desc_spkm->HasPrivKey(xpub.pubkey.GetID()));
-                }
-                if (priv) {
-                    if (std::optional<CExtKey> xprv = wallet->GetExtKey(xpub)) {
-                        wallet_xprvs[xpub] = *xprv;
+                    if (std::optional<CKey> key = priv ? desc_spkm->GetKey(xpub.pubkey.GetID()) : std::nullopt) {
+                        wallet_xprvs[xpub] = CExtKey{xpub, *key};
                     }
                 }
             }

Comment thread src/wallet/rpc/wallet.cpp
},
},
RPCExamples{
HelpExampleCli("derivehdkey", "m/87h/0h/0h") + HelpExampleRpc("derivehdkey", "")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter is required.

diff --git a/src/wallet/rpc/wallet.cpp b/src/wallet/rpc/wallet.cpp
index cfba900d22..33450695e1 100644
--- a/src/wallet/rpc/wallet.cpp
+++ b/src/wallet/rpc/wallet.cpp
@@ -979,7 +979,7 @@ RPCMethod derivehdkey()
             },
         },
         RPCExamples{
-            HelpExampleCli("derivehdkey", "m/87h/0h/0h") + HelpExampleRpc("derivehdkey", "")
+            HelpExampleCli("derivehdkey", "m/87h/0h/0h") + HelpExampleRpc("derivehdkey", "\"m/87h/0h/0h\"")
             + HelpExampleCliNamed("derivehdkey", {{"path", "m/87h/0h/0h"}, {"private", "true"}})
             + HelpExampleRpcNamed("derivehdkey", {{"path", "m/87h/0h/0h"}, {"private", "true"}})
         },

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants