Skip to content

766 select.dictionary_element: add split parameter - #996

Open
mborodii-prog wants to merge 1 commit into
mainfrom
766-selectdictionary_element-behavior-when-no-output-specified
Open

766 select.dictionary_element: add split parameter#996
mborodii-prog wants to merge 1 commit into
mainfrom
766-selectdictionary_element-behavior-when-no-output-specified

Conversation

@mborodii-prog

@mborodii-prog mborodii-prog commented May 26, 2026

Copy link
Copy Markdown
Contributor

select.dictionary_element: add split parameter

Summary

Per @thomasstvr's suggestion, instead of adding a new select.dict_element
wrangle, this adds a split parameter to the existing
select.dictionary_element wrangle. Default behavior is unchanged.

Motivation

select.dictionary_element overwrites the input column when no output
is specified, which is convenient for quick in-place replacements but
makes it easy to lose the original data (see #766). Rather than
introducing a separate wrangle with different default behavior, split
opts into the non-destructive behavior on the existing wrangle.

Behavior

split Behavior
false (default) Unchanged: writes to output if given, otherwise overwrites the input column
true Writes the selected element(s) to new column(s) named after the key(s), preserving the input column. Cannot be combined with output. With a single key, creates one column named after that key; a list, wildcard, or regex element creates one column per matched key.

Examples

Simple key — new column named after the element, input preserved:

wrangles:
- select.dictionary_element:
    input: product
    element: colour
    split: true
# input:  product = {'colour': 'red', 'shape': 'circle'}
# result: product = {'colour': 'red', 'shape': 'circle'}  (unchanged)
#         colour  = 'red'                                  (new)

List of keys — one column per key:

wrangles:
- select.dictionary_element:
    input: properties
    element:
    - A
    - B
    split: true
# input:  properties = {'A': '1', 'B': '2', 'C': '3'}  (unchanged)
# result: A = '1', B = '2'                               (new columns)

Wildcard — matched keys resolved at runtime, each becomes a column:

wrangles:
- select.dictionary_element:
    input: specs
    element: A*
    split: true
# input:  specs = {'A1': '1', 'B1': '2', 'A2': '3'}  (unchanged)
# result: A1 = '1', A2 = '3'                           (new columns)

Regex — same as wildcard:

wrangles:
- select.dictionary_element:
    input: specs
    element: 'regex: B.*'
    split: true
# input:  specs = {'A1': '1', 'B1': '2', 'A2': '3'}  (unchanged)
# result: B1 = '2'                                     (new column)

@mborodii-prog mborodii-prog linked an issue May 26, 2026 that may be closed by this pull request
@mborodii-prog
mborodii-prog marked this pull request as draft May 27, 2026 15:34
@thomasstvr

Copy link
Copy Markdown
Collaborator

Automatically outputting a split dictionary like this will be a breaking change. I would suggest leaving the default behavior as is and adding some sort of parameter that splits the selected elements.

@mborodii-prog

Copy link
Copy Markdown
Contributor Author

@thomasstvr it will not be a breaking change as it's new wrangle

@mborodii-prog
mborodii-prog marked this pull request as ready for review June 10, 2026 08:50
@thomasstvr

Copy link
Copy Markdown
Collaborator

@mborodii-prog somehow I completely missed that. In general, I don't think it is a good idea to just create a new wrangle when we don't like the behavior of another. I skipped right over it because the names are so similar myself.

@mborodii-prog mborodii-prog changed the title 766 selectdictionary element behavior when no output specified 766 select.dictionary_element: add split parameter Jun 15, 2026
@mborodii-prog

Copy link
Copy Markdown
Contributor Author

@thomasstvr I updated current PR approach that you proposed: added new parameter split instead of creating new wrangle

Adds a split parameter to select.dictionary_element. When split is
true, the selected element(s) are written to new column(s) named
after the key(s) instead of overwriting the input column, leaving the
input intact. Default behavior (split: false) is unchanged.
@mborodii-prog
mborodii-prog force-pushed the 766-selectdictionary_element-behavior-when-no-output-specified branch from e5a7acb to a2a0be3 Compare June 15, 2026 19:37
@ebhills

ebhills commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator

@thomasstvr - I don't think 'split' make senses in the context of select, especially since we have a split.dictionry.

While we generally don't want a breaking change, in this case I believe it is worthwhile. I think I ran a models table query that found all of the instances where this wrangle is used without an output. I will take the action item to rerun that and assess how we can change the behavior to be intuitive -> you specify the key(s) you want and they pop out as columns without having to specify output. That way a) it always yields columns, and b) the user only needs to specify output(s) when they want to rename.

I have never had a use for the current default behavior of 'selecting' a subset of keys but leaving it as a smaller dictionary. Have you?

@ebhills
ebhills removed their request for review July 27, 2026 14:04

ebhills commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Queue triage (2026-07-27)

  • Disposition: Ready for review
  • Delivery owner: @mborodii-prog
  • Primary reviewer: @thomasstvr
  • Next action: Review the split-parameter behavior and focused tests; approve or request blocking changes.

Please keep the branch current and put the decision in GitHub. This is one of the five active review slots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

select.dictionary_element behavior when no output specified

3 participants