diff --git a/CHANGELOG.md b/CHANGELOG.md index a22dc4fc..162b5f7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ### Fixed - Restore `py.typed` marker so type checkers recognize `hcl2` (and `cli`) as typed packages. ([#298](https://github.com/amplify-education/python-hcl2/issues/298)) +- `preserve_heredocs=False` combined with `strip_string_quotes` now returns the heredoc body as a plain multi-line string instead of escaping every newline to a literal `\n`. The escaping is still applied to the quoted source form produced without `strip_string_quotes`. ([#303](https://github.com/amplify-education/python-hcl2/issues/303)) - Parse heredocs whose delimiter is a single character, such as `<HCL2 deserialization and reconstruction.** | -| `preserve_heredocs` | `bool` | `True` | Keep heredocs in their original form | +| `preserve_heredocs` | `bool` | `True` | Keep heredocs in their original form, markers included. When `False`, a heredoc becomes a quoted string with its newlines escaped (`'"a\nb"'`); combine with `strip_string_quotes` to get the body as a plain multi-line value instead. | | `force_operation_parentheses` | `bool` | `False` | Force parentheses around all operations | | `preserve_scientific_notation` | `bool` | `True` | Keep scientific notation as-is | | `strip_string_quotes` | `bool` | `False` | Yield string *values* rather than source text: remove surrounding quotes (e.g. `"hello"` instead of `'"hello"'`) and resolve escape sequences (`"a\nb"` becomes a real newline). String literals inside expressions keep their quotes, so `upper("x")` stays `'${upper("x")}'`. **Breaks JSON->HCL2 deserialization and reconstruction.** | diff --git a/docs/06_migrating_to_v8.md b/docs/06_migrating_to_v8.md index d6383e0a..0c4703bf 100644 --- a/docs/06_migrating_to_v8.md +++ b/docs/06_migrating_to_v8.md @@ -201,9 +201,29 @@ V7_COMPAT = SerializationOptions( strip_string_quotes=True, explicit_blocks=False, with_comments=False, + preserve_heredocs=False, ) data = hcl2.load(f, serialization_options=V7_COMPAT) ``` This restores the v7 dict shape but disables round-trip support and comment preservation. + +`preserve_heredocs=False` matters if your configuration uses heredocs. Left at its default, a heredoc value keeps its `<<-EOT` / `EOT` markers embedded in the string. Turned off alongside `strip_string_quotes`, you get the body as a plain multi-line string with real line breaks, which is what v7 returned: + +```python +hcl2.loads('x = <<-EOT\n line1\n line2\n EOT\n', serialization_options=V7_COMPAT) +# {'x': 'line1\nline2'} +``` + +Note that `preserve_heredocs=False` on its own — without `strip_string_quotes` — produces the quoted *source* form with escaped newlines (`'"line1\\nline2"'`), because that output is meant to be reconstructable. + +Two details of heredoc values are easy to trip over, and both match how HCL itself behaves: + +- **Backslash escapes are not interpreted in heredocs.** `strip_string_quotes` resolves `\n` inside a *quoted* string, but a heredoc body containing the two characters `\n` keeps them verbatim. HCL only processes escape sequences in quoted templates. +- **Line endings come through as written.** A heredoc in a CRLF file yields a body with `\r\n`, because a carriage return inside the body is content rather than structure. Normalize on your side if you need `\n`. + +```python +hcl2.loads('x = <