v0.1.0 initial commit

This commit is contained in:
2026-07-13 02:45:49 -04:00
commit c8e23ff859
15 changed files with 4161 additions and 0 deletions
+8
View File
@@ -0,0 +1,8 @@
.idea
bin/
dev/
logs/
log/
examples/
catalog.txt
combined.txt
+28
View File
@@ -0,0 +1,28 @@
BSD 3-Clause License
Copyright (c) 2026, Lixen Wraith
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+154
View File
@@ -0,0 +1,154 @@
# toml
Zero-dependency TOML encoder/decoder for Go.
Implements a practical subset of TOML v1.0 with documented deviations.
Standard library only.
## Install
go get github.com/lixenwraith/toml
## Usage
### Unmarshal
```go
input := []byte(`
initial = "Idle"
[states.Idle]
parent = "Root"
transitions = [
{ trigger = "Start", target = "Active" }
]
`)
type Transition struct {
Trigger string `toml:"trigger"`
Target string `toml:"target"`
Guard string `toml:"guard,omitempty"`
}
type State struct {
Parent string `toml:"parent"`
Transitions []Transition `toml:"transitions"`
}
type Config struct {
Initial string `toml:"initial"`
States map[string]*State `toml:"states"`
}
var cfg Config
err := toml.Unmarshal(input, &cfg)
```
### Marshal
```go
out, err := toml.Marshal(cfg) // deterministic: keys sorted alphabetically
```
### Decode
`Decode(data any, v any)` maps an already-parsed `map[string]any` onto a
target. `Unmarshal` = `Parse` + `Decode`.
## Architecture
Unmarshal: []byte → Lexer → tokens → Parser → map[string]any → Decode (reflection) → target
Marshal: value → encoder (reflection, two-pass) → []byte
### Lexer (`lexer.go`)
- Single-pass, UTF-8 aware byte scanner with rune lookahead (`peekAt`).
- Emits: ident, string, integer, float, bool, punctuation, newline, comment.
- Number/key disambiguation happens at the token level: `1.5` → Float,
`a.b` → Ident Dot Ident, `1.2.3` → Error (multi-dot). Hex/octal/binary
prefixes and exponent forms are validated during scanning.
- Basic strings only. Escapes: `\"` `\\` `\n` `\t` `\r`. Unknown escape
sequences are preserved verbatim (spec requires an error — see Deviations).
### Parser (`parser.go`)
- Recursive descent with a two-token window (`curToken`/`peekToken`).
Comments are skipped during token advance.
- Output model: tables → `map[string]any`, arrays → `[]any`,
arrays of tables → `[]map[string]any`.
- Table headers (`[a.b]`) always resolve from the root; a `current` cursor
tracks the active table scope for subsequent key/value pairs.
- Dotted keys create intermediate maps. Traversal through an existing
`[[array]]` descends into its last element (TOML semantics).
- Conflict detection at assignment: duplicate keys, scalar-vs-table
redefinition, table-vs-array-of-tables shadowing. Numeric keys are
rejected in all positions (bare, quoted, dotted segments).
- Inline tables are immutable (later dotted keys or `[headers]` targeting them error); value nesting (arrays/inline tables) capped at 1000 levels.
### Decoder (`decode.go`)
- Kind-switch reflection. Pointers are auto-allocated at any depth
(`******int` works). Nil maps and slices are materialized.
- Struct fields match by `toml` tag first, then exact (case-sensitive)
field name. `toml:"-"` skips. Unexported fields are skipped.
- Numeric coercion: any parser-produced int/uint/float64 converts to the
target numeric kind. Unknown keys in input are ignored.
- Numeric decode errors on target overflow (`300``int8` errors); non-empty interface targets error unless assignable; unsupported target kinds (chan, func, complex, fixed arrays) error; `uint`/`uint64` sources above `MaxInt64` are rejected.
### Encoder (`encode.go`)
- Two-pass emission per table: scalars and inline arrays first, then
`[tables]` and `[[arrays of tables]]`. Guarantees keys are defined
before sub-tables, i.e. output is always valid TOML.
- Keys sorted alphabetically → deterministic output for diffing/VCS.
- Bare-key validation mirrors the lexer's rules: keys that would lex as
numbers or booleans (`true`, `123a`, `-1x`) are quoted, so output always
re-parses (round-trip safe).
- `omitempty` honored; nil pointers and `toml:"-"` fields skipped;
root must be a struct or map.
- Floats use shortest `'g'` form at the source bit width (large magnitudes emit exponent notation); `NaN`/`±Inf` return an error; arrays mixing tables and scalars return an error (parser accepts them — encode-side limitation); struct keys sort by resolved tag name.
## Type Mapping
| TOML | Parser output (`map[string]any`) | Decode targets |
|-----------------|----------------------------------|-----------------------------------|
| string | `string` | `string`, `any` |
| integer | `int` | any int/uint kind, float kinds |
| float | `float64` | `float32`, `float64` |
| boolean | `bool` | `bool` |
| array | `[]any` | `[]T`, `any` |
| table | `map[string]any` | struct, `map[string]T`, `any` |
| array of tables | `[]map[string]any` | `[]T`, `[]*T` |
## Deviations from TOML v1.0
Intentional restrictions:
- Numeric keys are forbidden everywhere: `123 = 1`, `[456]`, `"789"`,
`a.1.b` all error. Stricter than spec.
Extensions (accepted, though spec forbids):
- Newlines inside inline tables (multi-line inline tables parse).
- Trailing commas in inline tables.
Relaxations:
- Unknown string escapes are preserved instead of erroring.
- Explicit table redefinition (`[a]``[a]`) is not rejected; the second
header reopens the table. Value/table conflicts are still caught.
## Limitations
Not supported:
- Date/time types (all four TOML forms). Workaround: store RFC 3339
strings and convert with `time.Parse` at the call site; strings
round-trip cleanly.
- Literal strings `'...'` and multi-line strings `"""..."""`, `'''...'''`.
- Underscore digit separators (`1_000`).
- Arrays mixing tables and scalar values are not encodable.
- `inf` / `nan` float literals.
- Custom marshaling interfaces (`Marshaler`/`Unmarshaler`,
`encoding.TextMarshaler`).
- Embedded struct field promotion in decode.
Platform note: TOML integers parse to Go `int`; 64-bit platforms assumed (amd64/arm64) and enforced at compile time (32-bit build fails).
Overflow of int64 during parse errors; 32-bit truncation of `int(int64)` is not guarded.
## License
BSD-3-Clause (see LICENSE).
+200
View File
@@ -0,0 +1,200 @@
package toml
import (
"math"
"strings"
"testing"
)
func TestDecode_NumericOverflow(t *testing.T) {
type T struct {
I8 int8 `toml:"i8"`
U8 uint8 `toml:"u8"`
F32 float32 `toml:"f32"`
}
var tgt T
if err := Decode(map[string]any{"i8": 300}, &tgt); err == nil {
t.Errorf("300 into int8 must error, got %d", tgt.I8)
}
if err := Decode(map[string]any{"u8": 256}, &tgt); err == nil {
t.Errorf("256 into uint8 must error, got %d", tgt.U8)
}
if err := Decode(map[string]any{"f32": 1e300}, &tgt); err == nil {
t.Errorf("1e300 into float32 must error, got %g", tgt.F32)
}
// Boundaries remain valid
if err := Decode(map[string]any{"i8": 127, "u8": 255}, &tgt); err != nil {
t.Fatalf("boundary decode failed: %v", err)
}
if tgt.I8 != 127 || tgt.U8 != 255 {
t.Errorf("boundary values: %d %d", tgt.I8, tgt.U8)
}
}
func TestDecode_UnsupportedKinds(t *testing.T) {
type T struct {
C complex128 `toml:"v"`
}
var c T
if err := Decode(map[string]any{"v": 1}, &c); err == nil {
t.Error("complex128 target must error, not zero-fill")
}
type T2 struct {
Ch chan int `toml:"v"`
}
var c2 T2
if err := Decode(map[string]any{"v": 1}, &c2); err == nil {
t.Error("chan target must error")
}
}
func TestDecode_NonEmptyInterface(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("input-triggered panic: %v", r)
}
}()
type T struct {
R interface{ Read([]byte) (int, error) } `toml:"r"`
}
var tgt T
if err := Decode(map[string]any{"r": map[string]any{}}, &tgt); err == nil {
t.Error("map into non-empty interface must error")
}
}
func TestDecode_Uint64Wrap(t *testing.T) {
type T struct {
V int64 `toml:"v"`
}
var tgt T
if err := Decode(map[string]any{"v": uint64(math.MaxUint64)}, &tgt); err == nil {
t.Errorf("MaxUint64 into int64 must error, got %d", tgt.V)
}
if err := Decode(map[string]any{"v": uint(math.MaxUint64)}, &tgt); err == nil {
t.Errorf("uint wrap into int64 must error, got %d", tgt.V)
}
if err := Decode(map[string]any{"v": uint64(42)}, &tgt); err != nil || tgt.V != 42 {
t.Errorf("in-range uint64 failed: %v %d", err, tgt.V)
}
}
func TestMarshal_FloatFormatting(t *testing.T) {
b, err := Marshal(map[string]any{"f": float32(3.14)})
if err != nil {
t.Fatal(err)
}
if got := strings.TrimSpace(string(b)); got != "f = 3.14" {
t.Errorf("float32 noise digits: %s", got)
}
b, err = Marshal(map[string]any{"f": 1e100})
if err != nil {
t.Fatal(err)
}
if got := strings.TrimSpace(string(b)); got != "f = 1e+100" {
t.Errorf("digit expansion instead of exponent: %s", got)
}
var m map[string]any
if err := Unmarshal(b, &m); err != nil {
t.Fatalf("re-parse of exponent form failed: %v", err)
}
if m["f"] != 1e100 {
t.Errorf("round trip mismatch: %v", m["f"])
}
// Integral floats keep the .0 fixup
b, _ = Marshal(map[string]any{"f": 100.0})
if got := strings.TrimSpace(string(b)); got != "f = 100.0" {
t.Errorf(".0 fixup lost: %s", got)
}
}
func TestMarshal_NaNInf(t *testing.T) {
for _, v := range []float64{math.NaN(), math.Inf(1), math.Inf(-1)} {
if _, err := Marshal(map[string]any{"f": v}); err == nil {
t.Errorf("%v must fail to encode", v)
}
}
}
func TestMarshal_SliceHomogeneity(t *testing.T) {
if _, err := Marshal(map[string]any{"x": []any{map[string]any{"a": 1}, 2}}); err == nil {
t.Error("mixed table/scalar slice must error")
}
if _, err := Marshal(map[string]any{"x": []any{1, nil, 2}}); err == nil {
t.Error("nil interface element must error")
}
type Item struct {
N int `toml:"n"`
}
b, err := Marshal(map[string]any{"items": []*Item{nil, {N: 1}}})
if err != nil {
t.Fatalf("nil pointer in table slice must be skipped, not error: %v", err)
}
if strings.Count(string(b), "[[items]]") != 1 {
t.Errorf("expected exactly one [[items]]:\n%s", b)
}
}
func TestMarshal_TagSortedKeys(t *testing.T) {
type T struct {
B int `toml:"z"`
Z int `toml:"a"`
}
b, err := Marshal(T{B: 1, Z: 2})
if err != nil {
t.Fatal(err)
}
if got := strings.TrimSpace(string(b)); got != "a = 2\nz = 1" {
t.Errorf("keys not sorted by emitted name:\n%s", got)
}
}
func TestLexer_UnicodeEscapes(t *testing.T) {
var m map[string]any
if err := Unmarshal([]byte(`s = "caf\u00E9 \U0001F600"`), &m); err != nil {
t.Fatalf("unicode escape parse failed: %v", err)
}
if m["s"] != "café 😀" {
t.Errorf("got %q", m["s"])
}
for _, in := range []string{
`s = "\uD800"`, // surrogate
`s = "\u12"`, // too few digits
`s = "\U00110000"`, // > MaxRune
`s = "\uZZZZ"`, // non-hex
} {
if err := Unmarshal([]byte(in), &m); err == nil {
t.Errorf("%s should fail", in)
}
}
}
func TestParser_InlineTableImmutable(t *testing.T) {
for _, in := range []string{
"t = { a = 1 }\nt.b = 2",
"t = { a = 1 }\n[t]\nb = 2",
"t = { a = 1 }\n[t.sub]\nb = 2",
} {
p := NewParser([]byte(in))
if _, err := p.Parse(); err == nil {
t.Errorf("inline table extension should fail:\n%s", in)
}
}
// Intra-table dotted keys remain valid
var m map[string]any
if err := Unmarshal([]byte(`t = { a.b = 1, a.c = 2 }`), &m); err != nil {
t.Errorf("dotted keys within inline table must parse: %v", err)
}
}
func TestParser_ValueDepthLimit(t *testing.T) {
input := "x = " + strings.Repeat("[", 5000) + strings.Repeat("]", 5000)
p := NewParser([]byte(input))
if _, err := p.Parse(); err == nil {
t.Error("expected nesting depth error")
}
}
+253
View File
@@ -0,0 +1,253 @@
package toml
import (
"fmt"
"math"
"reflect"
"strings"
)
// Unmarshal parses TOML data and stores the result in the value pointed to by v.
func Unmarshal(data []byte, v any) error {
p := NewParser(data)
parsedMap, err := p.Parse()
if err != nil {
return err
}
return Decode(parsedMap, v)
}
// Decode maps a generic map[string]any to a struct/slice/etc using reflection.
// It prioritizes `toml` tags and falls back to field names.
func Decode(data any, v any) error {
val := reflect.ValueOf(v)
if val.Kind() != reflect.Ptr || val.IsNil() {
return fmt.Errorf("target must be a non-nil pointer")
}
return decodeValue(data, val.Elem())
}
func decodeValue(data any, val reflect.Value) error {
if data == nil {
return nil
}
switch val.Kind() {
case reflect.Ptr:
elemType := val.Type().Elem()
newVal := reflect.New(elemType)
if err := decodeValue(data, newVal.Elem()); err != nil {
return err
}
val.Set(newVal)
case reflect.Struct:
dataMap, ok := data.(map[string]any)
if !ok {
return fmt.Errorf("expected map for struct, got %T", data)
}
return decodeStruct(dataMap, val)
case reflect.Slice:
dataSlice, ok := data.([]any)
if !ok {
if mapSlice, ok := data.([]map[string]any); ok {
dataSlice = make([]any, len(mapSlice))
for i, m := range mapSlice {
dataSlice[i] = m
}
} else {
return fmt.Errorf("expected slice, got %T", data)
}
}
newSlice := reflect.MakeSlice(val.Type(), len(dataSlice), len(dataSlice))
for i := 0; i < len(dataSlice); i++ {
if err := decodeValue(dataSlice[i], newSlice.Index(i)); err != nil {
return err
}
}
val.Set(newSlice)
case reflect.Map:
if val.Type().Key().Kind() != reflect.String {
return fmt.Errorf("only map[string]T is supported")
}
dataMap, ok := data.(map[string]any)
if !ok {
return fmt.Errorf("expected map, got %T", data)
}
newMap := reflect.MakeMap(val.Type())
elemType := val.Type().Elem()
for k, vData := range dataMap {
newVal := reflect.New(elemType).Elem()
if err := decodeValue(vData, newVal); err != nil {
return fmt.Errorf("map key %s: %w", k, err)
}
newMap.SetMapIndex(reflect.ValueOf(k), newVal)
}
val.Set(newMap)
case reflect.Interface:
// Unchecked Set panics for non-empty interface targets (io.Reader etc.)
dv := reflect.ValueOf(data)
if !dv.Type().AssignableTo(val.Type()) {
return fmt.Errorf("cannot assign %T to interface %s", data, val.Type())
}
val.Set(dv)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
i, ok := toInt64(data)
if !ok {
return fmt.Errorf("cannot convert %T to int", data)
}
// SetInt truncates silently on narrower kinds
if val.OverflowInt(i) {
return fmt.Errorf("value %d overflows %s", i, val.Type())
}
val.SetInt(i)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
i, ok := toInt64(data)
if !ok {
return fmt.Errorf("cannot convert %T to uint", data)
}
if i < 0 {
return fmt.Errorf("cannot convert negative value %d to uint", i)
}
// Overflow check
if val.OverflowUint(uint64(i)) {
return fmt.Errorf("value %d overflows %s", i, val.Type())
}
val.SetUint(uint64(i))
case reflect.Float32, reflect.Float64:
f, ok := toFloat(data)
if !ok {
return fmt.Errorf("cannot convert %T to float", data)
}
// float64 -> float32 range check
if val.OverflowFloat(f) {
return fmt.Errorf("value %g overflows %s", f, val.Type())
}
val.SetFloat(f)
case reflect.String:
s, ok := data.(string)
if !ok {
return fmt.Errorf("cannot convert %T to string", data)
}
val.SetString(s)
case reflect.Bool:
b, ok := data.(bool)
if !ok {
return fmt.Errorf("cannot convert %T to bool", data)
}
val.SetBool(b)
default:
// Reject unsupported kinds instead of silent zero value
return fmt.Errorf("unsupported target kind %s", val.Kind())
}
return nil
}
func decodeStruct(data map[string]any, val reflect.Value) error {
typ := val.Type()
for i := 0; i < val.NumField(); i++ {
field := val.Field(i)
fieldType := typ.Field(i)
// Safety check: skip unexported fields that cannot be set
if !field.CanSet() {
continue
}
// Determine key name
key := fieldType.Name
if tag := fieldType.Tag.Get("toml"); tag != "" {
parts := strings.Split(tag, ",")
if parts[0] == "-" {
continue
}
key = parts[0]
}
// Look up in data map (case sensitive)
if vData, ok := data[key]; ok {
if err := decodeValue(vData, field); err != nil {
return fmt.Errorf("%s.%s: %w", typ.Name(), fieldType.Name, err)
}
}
}
return nil
}
// toInt64 converts numeric types to int64
func toInt64(v any) (int64, bool) {
switch i := v.(type) {
case int:
return int64(i), true
case int8:
return int64(i), true
case int16:
return int64(i), true
case int32:
return int64(i), true
case int64:
return i, true
case uint:
// Reject wrap
if uint64(i) > math.MaxInt64 {
return 0, false
}
case uint8:
return int64(i), true
case uint16:
return int64(i), true
case uint32:
return int64(i), true
case uint64:
// Reject values that wrap negative through int64
if i > math.MaxInt64 {
return 0, false
}
return int64(i), true
case float64:
return int64(i), true
}
return 0, false
}
func toFloat(v any) (float64, bool) {
switch i := v.(type) {
case int:
return float64(i), true
case int8:
return float64(i), true
case int16:
return float64(i), true
case int32:
return float64(i), true
case int64:
return float64(i), true
case uint:
return float64(i), true
case uint8:
return float64(i), true
case uint16:
return float64(i), true
case uint32:
return float64(i), true
case uint64:
return float64(i), true
case float64:
return i, true
}
return 0, false
}
+844
View File
@@ -0,0 +1,844 @@
package toml
import (
"testing"
)
// TestDecode_MapPointerValues tests map[string]*Struct decoding
// This is the exact pattern used by RootConfig.States
func TestDecode_MapPointerValues(t *testing.T) {
data := map[string]any{
"items": map[string]any{
"first": map[string]any{
"name": "alpha",
},
"second": map[string]any{
"name": "beta",
},
},
}
type Item struct {
Name string `toml:"name"`
}
type Config struct {
Items map[string]*Item `toml:"items"`
}
var cfg Config
if err := Decode(data, &cfg); err != nil {
t.Fatalf("Decode failed: %v", err)
}
if cfg.Items == nil {
t.Fatal("Items map is nil")
}
if len(cfg.Items) != 2 {
t.Fatalf("Expected 2 items, got %d", len(cfg.Items))
}
if cfg.Items["first"] == nil || cfg.Items["first"].Name != "alpha" {
t.Errorf("first item mismatch: %+v", cfg.Items["first"])
}
if cfg.Items["second"] == nil || cfg.Items["second"].Name != "beta" {
t.Errorf("second item mismatch: %+v", cfg.Items["second"])
}
}
// TestUnmarshal_DottedTableToMapPointer tests [parent.child] -> map[string]*Struct
func TestUnmarshal_DottedTableToMapPointer(t *testing.T) {
input := []byte(`
[states.Gameplay]
parent = "Root"
[states.TrySpawn]
parent = "Gameplay"
`)
type StateConfig struct {
Parent string `toml:"parent"`
}
type Config struct {
States map[string]*StateConfig `toml:"states"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.States == nil {
t.Fatal("States map is nil")
}
if len(cfg.States) != 2 {
t.Fatalf("Expected 2 states, got %d", len(cfg.States))
}
if cfg.States["Gameplay"] == nil {
t.Fatal("Gameplay state is nil")
}
if cfg.States["Gameplay"].Parent != "Root" {
t.Errorf("Gameplay.Parent mismatch: %q", cfg.States["Gameplay"].Parent)
}
}
// TestUnmarshal_InlineTableArray tests arrays of inline tables
func TestUnmarshal_InlineTableArray(t *testing.T) {
input := []byte(`
[state]
transitions = [
{ trigger = "EventA", target = "StateB" },
{ trigger = "EventB", target = "StateC", guard = "CheckX" }
]
`)
type Transition struct {
Trigger string `toml:"trigger"`
Target string `toml:"target"`
Guard string `toml:"guard,omitempty"`
}
type State struct {
Transitions []Transition `toml:"transitions"`
}
type Config struct {
State State `toml:"state"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if len(cfg.State.Transitions) != 2 {
t.Fatalf("Expected 2 transitions, got %d", len(cfg.State.Transitions))
}
if cfg.State.Transitions[0].Trigger != "EventA" {
t.Errorf("Transition[0].Trigger mismatch: %q", cfg.State.Transitions[0].Trigger)
}
if cfg.State.Transitions[1].Guard != "CheckX" {
t.Errorf("Transition[1].Guard mismatch: %q", cfg.State.Transitions[1].Guard)
}
}
func TestUnmarshal_MultilineInlineTable(t *testing.T) {
input := []byte(`
[state]
config = {
name = "test",
nested = { a = 1, b = 2 },
array = [
{ x = 10 },
{ x = 20 }
]
}
`)
type Inner struct {
X int `toml:"x"`
}
type Config struct {
Name string `toml:"name"`
Nested map[string]int `toml:"nested"`
Array []Inner `toml:"array"`
}
type State struct {
Config Config `toml:"config"`
}
type Root struct {
State State `toml:"state"`
}
var cfg Root
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.State.Config.Name != "test" {
t.Errorf("Name = %q", cfg.State.Config.Name)
}
if cfg.State.Config.Nested["a"] != 1 {
t.Errorf("Nested.a = %d", cfg.State.Config.Nested["a"])
}
if len(cfg.State.Config.Array) != 2 || cfg.State.Config.Array[1].X != 20 {
t.Errorf("Array = %+v", cfg.State.Config.Array)
}
}
func TestUnmarshal_DeeplyNestedMultiline(t *testing.T) {
input := []byte(`
transition = { trigger = "Tick", target = "Active", guard = "Or", guard_args = { guards = [
{ name = "Compare", args = { key = "val", op = "gt", value = 0 } },
{ name = "Check", args = { flag = true } }
]} }
`)
type Args struct {
Key string `toml:"key"`
Op string `toml:"op"`
Value int `toml:"value"`
Flag bool `toml:"flag"`
}
type Guard struct {
Name string `toml:"name"`
Args Args `toml:"args"`
}
type GuardArgs struct {
Guards []Guard `toml:"guards"`
}
type Transition struct {
Trigger string `toml:"trigger"`
Target string `toml:"target"`
Guard string `toml:"guard"`
GuardArgs GuardArgs `toml:"guard_args"`
}
type Root struct {
Transition Transition `toml:"transition"`
}
var cfg Root
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.Transition.Guard != "Or" {
t.Errorf("Guard = %q", cfg.Transition.Guard)
}
if len(cfg.Transition.GuardArgs.Guards) != 2 {
t.Fatalf("Guards count = %d", len(cfg.Transition.GuardArgs.Guards))
}
if cfg.Transition.GuardArgs.Guards[0].Args.Op != "gt" {
t.Errorf("Guards[0].Args.Op = %q", cfg.Transition.GuardArgs.Guards[0].Args.Op)
}
}
// TestUnmarshal_FSMConfigExact tests the exact FSM config structure
func TestUnmarshal_FSMConfigExact(t *testing.T) {
input := []byte(`
initial = "TrySpawnGold"
[states.Gameplay]
parent = "Root"
[states.TrySpawnGold]
parent = "Gameplay"
on_enter = [
{ action = "EmitEvent", event = "EventGoldSpawnRequest" }
]
transitions = [
{ trigger = "EventGoldSpawned", target = "GoldActive" },
{ trigger = "EventGoldSpawnFailed", target = "GoldRetryWait" }
]
[states.GoldRetryWait]
parent = "Gameplay"
transitions = [
{ trigger = "Tick", target = "TrySpawnGold", guard = "StateTimeExceeds", guard_args = { ms = 2000 } }
]
[states.GoldActive]
parent = "Gameplay"
transitions = [
{ trigger = "EventGoldCollected", target = "TrySpawnGold" }
]
`)
type ActionConfig struct {
Action string `toml:"action"`
Event string `toml:"event,omitempty"`
}
type TransitionConfig struct {
Trigger string `toml:"trigger"`
Target string `toml:"target"`
Guard string `toml:"guard,omitempty"`
GuardArgs map[string]any `toml:"guard_args,omitempty"`
}
type StateConfig struct {
Parent string `toml:"parent,omitempty"`
OnEnter []ActionConfig `toml:"on_enter,omitempty"`
Transitions []TransitionConfig `toml:"transitions,omitempty"`
}
type RootConfig struct {
InitialState string `toml:"initial"`
States map[string]*StateConfig `toml:"states"`
}
var cfg RootConfig
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
// Check initial
if cfg.InitialState != "TrySpawnGold" {
t.Errorf("InitialState mismatch: %q", cfg.InitialState)
}
// Check states map
if cfg.States == nil {
t.Fatal("States map is nil")
}
if len(cfg.States) != 4 {
t.Errorf("Expected 4 states, got %d", len(cfg.States))
for k := range cfg.States {
t.Logf(" Found state: %q", k)
}
}
// Check TrySpawnGold
tsg := cfg.States["TrySpawnGold"]
if tsg == nil {
t.Fatal("TrySpawnGold state is nil")
}
if tsg.Parent != "Gameplay" {
t.Errorf("TrySpawnGold.Parent mismatch: %q", tsg.Parent)
}
if len(tsg.OnEnter) != 1 {
t.Errorf("TrySpawnGold.OnEnter count mismatch: %d", len(tsg.OnEnter))
}
if len(tsg.Transitions) != 2 {
t.Errorf("TrySpawnGold.Transitions count mismatch: %d", len(tsg.Transitions))
}
// Check GoldRetryWait guard_args
grw := cfg.States["GoldRetryWait"]
if grw == nil {
t.Fatal("GoldRetryWait state is nil")
}
if len(grw.Transitions) != 1 {
t.Fatalf("GoldRetryWait.Transitions count mismatch: %d", len(grw.Transitions))
}
if grw.Transitions[0].GuardArgs == nil {
t.Error("GuardArgs is nil")
} else if ms, ok := grw.Transitions[0].GuardArgs["ms"]; !ok {
t.Error("GuardArgs missing 'ms' key")
} else if msInt, ok := ms.(int); !ok || msInt != 2000 {
t.Errorf("GuardArgs.ms mismatch: %T %v", ms, ms)
}
}
// TestParser_DottedTableStructure verifies parser output for dotted tables
func TestParser_DottedTableStructure(t *testing.T) {
input := []byte(`
[states.Alpha]
name = "first"
[states.Beta]
name = "second"
`)
p := NewParser(input)
result, err := p.Parse()
if err != nil {
t.Fatalf("Parse failed: %v", err)
}
// Check raw parser output structure
states, ok := result["states"]
if !ok {
t.Fatal("'states' key missing from parser output")
}
statesMap, ok := states.(map[string]any)
if !ok {
t.Fatalf("'states' is not map[string]any, got %T", states)
}
if len(statesMap) != 2 {
t.Errorf("Expected 2 states in parser output, got %d", len(statesMap))
}
alpha, ok := statesMap["Alpha"]
if !ok {
t.Error("'Alpha' key missing")
}
alphaMap, ok := alpha.(map[string]any)
if !ok {
t.Fatalf("'Alpha' is not map[string]any, got %T", alpha)
}
if alphaMap["name"] != "first" {
t.Errorf("Alpha.name mismatch: %v", alphaMap["name"])
}
}
// TestDecode_MapNilInitialization verifies map initialization during decode
func TestDecode_MapNilInitialization(t *testing.T) {
data := map[string]any{
"items": map[string]any{
"a": map[string]any{"val": 1},
},
}
type Item struct {
Val int `toml:"val"`
}
type Config struct {
Items map[string]*Item `toml:"items"` // nil initially
}
var cfg Config
// cfg.Items is nil here
if err := Decode(data, &cfg); err != nil {
t.Fatalf("Decode failed: %v", err)
}
if cfg.Items == nil {
t.Fatal("Decode did not initialize nil map")
}
}
func TestUnmarshal_ExtremeComplexity(t *testing.T) {
input := []byte(`
# Root level mixed types
version = "2.0.0-beta"
debug = true
tick_rate = 144
delta_time = 0.00694
# Deep dotted header (5 levels)
[engine.renderer.pipeline.stage.config]
name = "deferred"
priority = 1
enabled = true
scale_factor = 1.5e-2
tags = ["lighting", "shadows", "post-fx"]
# Nested inline table inside dotted section
[engine.renderer.pipeline.stage.config.viewport]
width = 1920
height = 1080
settings = { vsync = true, hdr = false, gamma = 2.2 }
# Hyphenated keys at multiple levels
[engine.audio-system.spatial-audio]
enabled = true
max-sources = 64
falloff-curve = "exponential"
rolloff-factor = 1.0e+0
# Map with pointer values using dotted headers
[game.entities.player]
health = 100
position.x = 0.0
position.y = -9.81e-1
position.z = 0.0
tags = ["controllable", "damageable"]
inventory = { slots = 20, weight_limit = 150.5 }
[game.entities.enemy-boss]
health = 5000
position.x = 100.0
position.y = 0.0
position.z = -50.0
tags = ["hostile", "boss", "damageable"]
ai = { aggression = 0.9, patrol_radius = 25 }
[game.entities."فارسی-test"]
health = 1
position.x = 1.0
position.y = 1.0
position.z = 1.0
tags = []
# Nested map of maps
[game.levels.level-01.zones.spawn-area]
bounds.min.x = -10
bounds.min.y = 0
bounds.min.z = -10
bounds.max.x = 10
bounds.max.y = 5
bounds.max.z = 10
enemy_count = 0
is_safe = true
[game.levels.level-01.zones.combat-zone]
bounds.min.x = 50
bounds.min.y = 0
bounds.min.z = 50
bounds.max.x = 150
bounds.max.y = 20
bounds.max.z = 150
enemy_count = 25
is_safe = false
# Array of tables with nested complexity
[[game.waves]]
id = 1
delay_ms = 0
spawns = [
{ entity = "enemy-grunt", count = 5, position = { x = 10.0, y = 0.0, z = 10.0 } },
{ entity = "enemy-scout", count = 3, position = { x = -10.0, y = 0.0, z = 10.0 } }
]
[[game.waves]]
id = 2
delay_ms = 30000
spawns = [
{ entity = "enemy-boss", count = 1, position = { x = 0.0, y = 0.0, z = 50.0 } }
]
# Deeply nested with mixed inline and standard tables
[physics.collision.layers.player-projectiles]
mask = 0b1010
priority = 10
callbacks.on_enter = "HandleProjectileHit"
callbacks.on_exit = "CleanupProjectile"
[physics.collision.layers.environment]
mask = 0b1111
priority = 1
callbacks.on_enter = "HandleCollision"
callbacks.on_exit = ""
# Scientific notation stress test
[constants]
planck = 6.62607015e-34
c = 2.998e+8
epsilon_0 = 8.854e-12
very_small = 1e-100
very_large = 1e+100
negative_exp = -5.5e-10
# Empty and edge cases mixed in
[edge.cases]
empty_string = ""
empty_array = []
empty_inline = {}
zero_int = 0
zero_float = 0.0
negative_int = -42
negative_float = -273.15
unicode_value = "日本語テスト 🎮 Ελληνικά"
hex_val = 0xDEAD
octal_val = 0o755
binary_val = 0b1010
`)
type Vec3 struct {
X float64 `toml:"x"`
Y float64 `toml:"y"`
Z float64 `toml:"z"`
}
type Bounds struct {
Min Vec3 `toml:"min"`
Max Vec3 `toml:"max"`
}
type ViewportSettings struct {
Vsync bool `toml:"vsync"`
HDR bool `toml:"hdr"`
Gamma float64 `toml:"gamma"`
}
type Viewport struct {
Width int `toml:"width"`
Height int `toml:"height"`
Settings ViewportSettings `toml:"settings"`
}
type StageConfig struct {
Name string `toml:"name"`
Priority int `toml:"priority"`
Enabled bool `toml:"enabled"`
ScaleFactor float64 `toml:"scale_factor"`
Tags []string `toml:"tags"`
Viewport Viewport `toml:"viewport"`
}
type Stage struct {
Config StageConfig `toml:"config"`
}
type Pipeline struct {
Stage Stage `toml:"stage"`
}
type Renderer struct {
Pipeline Pipeline `toml:"pipeline"`
}
type SpatialAudio struct {
Enabled bool `toml:"enabled"`
MaxSources int `toml:"max-sources"`
FalloffCurve string `toml:"falloff-curve"`
RolloffFactor float64 `toml:"rolloff-factor"`
}
type AudioSystem struct {
SpatialAudio SpatialAudio `toml:"spatial-audio"`
}
type Engine struct {
Renderer Renderer `toml:"renderer"`
AudioSystem AudioSystem `toml:"audio-system"`
}
type EntityConfig struct {
Health int `toml:"health"`
Position Vec3 `toml:"position"`
Tags []string `toml:"tags"`
Inventory map[string]any `toml:"inventory,omitempty"`
AI map[string]any `toml:"ai,omitempty"`
}
type Zone struct {
Bounds Bounds `toml:"bounds"`
EnemyCount int `toml:"enemy_count"`
IsSafe bool `toml:"is_safe"`
}
type Level struct {
Zones map[string]*Zone `toml:"zones"`
}
type SpawnPoint struct {
Entity string `toml:"entity"`
Count int `toml:"count"`
Position map[string]any `toml:"position"`
}
type Wave struct {
ID int `toml:"id"`
DelayMs int `toml:"delay_ms"`
Spawns []SpawnPoint `toml:"spawns"`
}
type Game struct {
Entities map[string]*EntityConfig `toml:"entities"`
Levels map[string]*Level `toml:"levels"`
Waves []*Wave `toml:"waves"`
}
type Callbacks struct {
OnEnter string `toml:"on_enter"`
OnExit string `toml:"on_exit"`
}
type CollisionLayer struct {
Mask int `toml:"mask"`
Priority int `toml:"priority"`
Callbacks Callbacks `toml:"callbacks"`
}
type Collision struct {
Layers map[string]*CollisionLayer `toml:"layers"`
}
type Physics struct {
Collision Collision `toml:"collision"`
}
type Constants struct {
Planck float64 `toml:"planck"`
C float64 `toml:"c"`
Epsilon0 float64 `toml:"epsilon_0"`
VerySmall float64 `toml:"very_small"`
VeryLarge float64 `toml:"very_large"`
NegativeExp float64 `toml:"negative_exp"`
}
type EdgeCases struct {
EmptyString string `toml:"empty_string"`
EmptyArray []any `toml:"empty_array"`
EmptyInline map[string]any `toml:"empty_inline"`
ZeroInt int `toml:"zero_int"`
ZeroFloat float64 `toml:"zero_float"`
NegativeInt int `toml:"negative_int"`
NegativeFloat float64 `toml:"negative_float"`
UnicodeValue string `toml:"unicode_value"`
HexVal int `toml:"hex_val"`
OctalVal int `toml:"octal_val"`
BinaryVal int `toml:"binary_val"`
}
type Edge struct {
Cases EdgeCases `toml:"cases"`
}
type Config struct {
Version string `toml:"version"`
Debug bool `toml:"debug"`
TickRate int `toml:"tick_rate"`
DeltaTime float64 `toml:"delta_time"`
Engine Engine `toml:"engine"`
Game Game `toml:"game"`
Physics Physics `toml:"physics"`
Constants Constants `toml:"constants"`
Edge Edge `toml:"edge"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
// Root level
if cfg.Version != "2.0.0-beta" {
t.Errorf("Version = %q", cfg.Version)
}
if !cfg.Debug {
t.Error("Debug should be true")
}
if cfg.TickRate != 144 {
t.Errorf("TickRate = %d", cfg.TickRate)
}
// 5-level deep dotted header
sc := cfg.Engine.Renderer.Pipeline.Stage.Config
if sc.Name != "deferred" {
t.Errorf("Stage.Config.Name = %q", sc.Name)
}
if sc.ScaleFactor != 1.5e-2 {
t.Errorf("ScaleFactor = %e", sc.ScaleFactor)
}
if len(sc.Tags) != 3 || sc.Tags[1] != "shadows" {
t.Errorf("Stage tags = %v", sc.Tags)
}
if sc.Viewport.Width != 1920 {
t.Errorf("Viewport.Width = %d", sc.Viewport.Width)
}
if sc.Viewport.Settings.Gamma != 2.2 {
t.Errorf("Viewport.Settings.Gamma = %f", sc.Viewport.Settings.Gamma)
}
// Hyphenated keys
sa := cfg.Engine.AudioSystem.SpatialAudio
if sa.MaxSources != 64 {
t.Errorf("MaxSources = %d", sa.MaxSources)
}
if sa.FalloffCurve != "exponential" {
t.Errorf("FalloffCurve = %q", sa.FalloffCurve)
}
// Map pointer values with dotted keys inside
player := cfg.Game.Entities["player"]
if player == nil {
t.Fatal("player entity nil")
}
if player.Health != 100 {
t.Errorf("player.Health = %d", player.Health)
}
if player.Position.Y != -9.81e-1 {
t.Errorf("player.Positions.Y = %e", player.Position.Y)
}
if len(player.Tags) != 2 {
t.Errorf("player.Tags = %v", player.Tags)
}
boss := cfg.Game.Entities["enemy-boss"]
if boss == nil {
t.Fatal("enemy-boss entity nil")
}
if boss.Health != 5000 {
t.Errorf("boss.Health = %d", boss.Health)
}
// Unicode key (edge case)
unicode := cfg.Game.Entities["فارسی-test"]
if unicode == nil {
t.Fatal("unicode entity nil")
}
if unicode.Health != 1 {
t.Errorf("unicode.Health = %d", unicode.Health)
}
// Deeply nested map of maps
lvl := cfg.Game.Levels["level-01"]
if lvl == nil {
t.Fatal("level-01 nil")
}
spawn := lvl.Zones["spawn-area"]
if spawn == nil {
t.Fatal("spawn-area nil")
}
if spawn.Bounds.Min.X != -10 {
t.Errorf("spawn.Bounds.Min.X = %f", spawn.Bounds.Min.X)
}
if spawn.Bounds.Max.Y != 5 {
t.Errorf("spawn.Bounds.Max.Y = %f", spawn.Bounds.Max.Y)
}
if !spawn.IsSafe {
t.Error("spawn.IsSafe should be true")
}
combat := lvl.Zones["combat-zone"]
if combat == nil {
t.Fatal("combat-zone nil")
}
if combat.EnemyCount != 25 {
t.Errorf("combat.EnemyCount = %d", combat.EnemyCount)
}
// Array of tables with pointer slice
if len(cfg.Game.Waves) != 2 {
t.Fatalf("Waves count = %d", len(cfg.Game.Waves))
}
w1 := cfg.Game.Waves[0]
if w1.ID != 1 || w1.DelayMs != 0 {
t.Errorf("Wave[0] = %+v", w1)
}
if len(w1.Spawns) != 2 {
t.Errorf("Wave[0].Spawns count = %d", len(w1.Spawns))
}
if w1.Spawns[0].Entity != "enemy-grunt" || w1.Spawns[0].Count != 5 {
t.Errorf("Wave[0].Spawns[0] = %+v", w1.Spawns[0])
}
w2 := cfg.Game.Waves[1]
if w2.DelayMs != 30000 {
t.Errorf("Wave[1].DelayMs = %d", w2.DelayMs)
}
// Collision layers map
projLayer := cfg.Physics.Collision.Layers["player-projectiles"]
if projLayer == nil {
t.Fatal("player-projectiles layer nil")
}
if projLayer.Mask != 0b1010 {
t.Errorf("projLayer.Mask = %d", projLayer.Mask)
}
if projLayer.Callbacks.OnEnter != "HandleProjectileHit" {
t.Errorf("projLayer.Callbacks.OnEnter = %q", projLayer.Callbacks.OnEnter)
}
// Scientific notation
if cfg.Constants.Planck != 6.62607015e-34 {
t.Errorf("Planck = %e", cfg.Constants.Planck)
}
if cfg.Constants.C != 2.998e+8 {
t.Errorf("C = %e", cfg.Constants.C)
}
if cfg.Constants.VerySmall != 1e-100 {
t.Errorf("VerySmall = %e", cfg.Constants.VerySmall)
}
if cfg.Constants.NegativeExp != -5.5e-10 {
t.Errorf("NegativeExp = %e", cfg.Constants.NegativeExp)
}
// Edge cases
if cfg.Edge.Cases.EmptyString != "" {
t.Errorf("EmptyString = %q", cfg.Edge.Cases.EmptyString)
}
if len(cfg.Edge.Cases.EmptyArray) != 0 {
t.Errorf("EmptyArray = %v", cfg.Edge.Cases.EmptyArray)
}
if len(cfg.Edge.Cases.EmptyInline) != 0 {
t.Errorf("EmptyInline = %v", cfg.Edge.Cases.EmptyInline)
}
if cfg.Edge.Cases.NegativeInt != -42 {
t.Errorf("NegativeInt = %d", cfg.Edge.Cases.NegativeInt)
}
if cfg.Edge.Cases.NegativeFloat != -273.15 {
t.Errorf("NegativeFloat = %f", cfg.Edge.Cases.NegativeFloat)
}
if cfg.Edge.Cases.UnicodeValue != "日本語テスト 🎮 Ελληνικά" {
t.Errorf("UnicodeValue = %q", cfg.Edge.Cases.UnicodeValue)
}
if cfg.Edge.Cases.HexVal != 0xDEAD {
t.Errorf("HexVal = %d, want %d", cfg.Edge.Cases.HexVal, 0xDEAD)
}
if cfg.Edge.Cases.OctalVal != 0o755 {
t.Errorf("OctalVal = %d, want %d", cfg.Edge.Cases.OctalVal, 0o755)
}
if cfg.Edge.Cases.BinaryVal != 0b1010 {
t.Errorf("BinaryVal = %d, want %d", cfg.Edge.Cases.BinaryVal, 0b1010)
}
}
+461
View File
@@ -0,0 +1,461 @@
package toml
import (
"bytes"
"fmt"
"math"
"reflect"
"sort"
"strconv"
"strings"
)
// Marshal returns the TOML encoding of v
//
// Marshal supports struct and map types as the root object
// It follows standard TOML formatting:
// - Comments and whitespace are not preserved from original decoding
// - Dates are not supported (as per requirements).
// - Nil pointers are skipped
// - Unexported fields are skipped
// - Fields with `omitempty` are skipped if zero
// - Struct fields/Map keys are sorted alphabetically for determinism
// - Fully numerical keys are rejected
func Marshal(v any) ([]byte, error) {
val := reflect.ValueOf(v)
// Dereference pointer if necessary
if val.Kind() == reflect.Ptr {
if val.IsNil() {
return nil, fmt.Errorf("marshal: cannot marshal nil pointer")
}
val = val.Elem()
}
// Root must be a Table (Struct or Map)
if val.Kind() != reflect.Struct && val.Kind() != reflect.Map {
return nil, fmt.Errorf("marshal: root must be struct or map, got %v", val.Kind())
}
buf := new(bytes.Buffer)
enc := &encoder{w: buf}
if err := enc.encodeTable(val, ""); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
type encoder struct {
w *bytes.Buffer
}
// encodeTable writes the fields of a struct or map.
// It uses a two-pass approach:
// 1. Write all scalar values (primitives, inline arrays).
// 2. Recurse and write nested tables (structs, maps, arrays of tables).
// This ensures valid TOML where keys are defined before sub-tables.
func (e *encoder) encodeTable(rv reflect.Value, prefix string) error {
// 1. Gather keys
keys, err := e.getSortedKeys(rv)
if err != nil {
return err
}
// 2. Separation: Identify which keys are scalars (printed now) vs tables (printed later)
var scalars []string
var tables []string
for _, k := range keys {
fieldVal := e.resolveValue(rv, k)
if !fieldVal.IsValid() {
continue // Skip invalid/nil
}
// Check if we should skip (omitempty, unexported handled in getSortedKeys)
if e.shouldSkip(rv, k, fieldVal) {
continue
}
// isTable validates slice homogeneity
isTab, err := e.isTable(fieldVal)
if err != nil {
return fmt.Errorf("key %q: %w", e.getKeyName(rv, k), err)
}
if isTab {
tables = append(tables, k)
} else {
scalars = append(scalars, k)
}
}
// 3. Pass 1: Write Scalars
for _, k := range scalars {
val := e.resolveValue(rv, k)
keyName := e.getKeyName(rv, k)
if err := e.writeKey(keyName); err != nil {
return err
}
if _, err := e.w.WriteString(" = "); err != nil {
return err
}
if err := e.encodeValue(val); err != nil {
return fmt.Errorf("key %q: %w", keyName, err)
}
e.w.WriteString("\n")
}
// 4. Pass 2: Write Tables
for _, k := range tables {
val := e.resolveValue(rv, k)
keyName := e.getKeyName(rv, k)
// Determine full path for header
fullKey := keyName
if prefix != "" {
fullKey = prefix + "." + keyName
}
// Handle specific table types
switch val.Kind() {
case reflect.Struct, reflect.Map:
// [header]
e.w.WriteString("\n")
e.w.WriteString("[" + fullKey + "]\n")
if err := e.encodeTable(val, fullKey); err != nil {
return err
}
case reflect.Slice, reflect.Array:
// [[header]]
for i := 0; i < val.Len(); i++ {
elem := val.Index(i)
// Dereference pointer elements in slice
if elem.Kind() == reflect.Ptr {
if elem.IsNil() {
continue
}
elem = elem.Elem()
}
e.w.WriteString("\n")
e.w.WriteString("[[" + fullKey + "]]\n")
if err := e.encodeTable(elem, fullKey); err != nil {
return err
}
}
}
}
return nil
}
// encodeValue writes a single primitive value or inline array
func (e *encoder) encodeValue(v reflect.Value) error {
switch v.Kind() {
case reflect.Bool:
if v.Bool() {
e.w.WriteString("true")
} else {
e.w.WriteString("false")
}
case reflect.String:
e.encodeString(v.String())
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
e.w.WriteString(strconv.FormatInt(v.Int(), 10))
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
e.w.WriteString(strconv.FormatUint(v.Uint(), 10))
case reflect.Float32, reflect.Float64:
f := v.Float()
// NaN/Inf have no TOML representation; emitting them produces
// output the parser rejects
if math.IsNaN(f) || math.IsInf(f, 0) {
return fmt.Errorf("cannot encode %v as TOML float", f)
}
// bitSize matches kind (float32 emitted noise digits);
// 'g' avoids ~100-digit 'f' expansions for large exponents
bits := 64
if v.Kind() == reflect.Float32 {
bits = 32
}
str := strconv.FormatFloat(f, 'g', -1, bits)
if !strings.ContainsAny(str, ".eE") {
str += ".0"
}
e.w.WriteString(str)
case reflect.Slice, reflect.Array:
// Inline array: [1, 2, "3"]
e.w.WriteString("[")
for i := 0; i < v.Len(); i++ {
if i > 0 {
e.w.WriteString(", ")
}
if err := e.encodeValue(v.Index(i)); err != nil {
return err
}
}
e.w.WriteString("]")
case reflect.Interface:
if v.IsNil() {
return nil // Should be handled by caller usually
}
return e.encodeValue(v.Elem())
default:
return fmt.Errorf("unsupported type: %v", v.Kind())
}
return nil
}
// --- Helpers ---
// getSortedKeys returns all field names (struct) or keys (map) sorted
func (e *encoder) getSortedKeys(rv reflect.Value) ([]string, error) {
var keys []string
if rv.Kind() == reflect.Map {
for _, key := range rv.MapKeys() {
if key.Kind() != reflect.String {
return nil, fmt.Errorf("map key must be string, got %v", key.Kind())
}
keys = append(keys, key.String())
}
} else if rv.Kind() == reflect.Struct {
typ := rv.Type()
for i := 0; i < rv.NumField(); i++ {
field := typ.Field(i)
// Skip unexported
if field.PkgPath != "" {
continue
}
// Skip if tag is "-"
tag := field.Tag.Get("toml")
if tag == "-" {
continue
}
keys = append(keys, field.Name)
}
}
// Sort by emitted key (tag-resolved), not Go field name.
// Identity for maps (getKeyName returns the key itself).
sort.Slice(keys, func(i, j int) bool {
return e.getKeyName(rv, keys[i]) < e.getKeyName(rv, keys[j])
})
return keys, nil
}
// resolveValue extracts the value from struct field or map key
// It also handles interface unwrapping for map[string]any.
func (e *encoder) resolveValue(container reflect.Value, key string) reflect.Value {
var val reflect.Value
if container.Kind() == reflect.Map {
val = container.MapIndex(reflect.ValueOf(key))
} else {
val = container.FieldByName(key)
}
// Unwrap interface if needed
if val.Kind() == reflect.Interface && !val.IsNil() {
val = val.Elem()
}
// Dereference pointer if needed (but keep nil ptrs for checking)
if val.Kind() == reflect.Ptr && !val.IsNil() {
val = val.Elem()
}
return val
}
// getKeyName resolves the TOML key name (handles struct tags)
// For maps, the key name is the key itself
func (e *encoder) getKeyName(container reflect.Value, realName string) string {
if container.Kind() == reflect.Map {
return realName
}
// Struct: lookup tag
field, _ := container.Type().FieldByName(realName)
tag := field.Tag.Get("toml")
if tag != "" {
parts := strings.Split(tag, ",")
if parts[0] != "" {
return parts[0]
}
}
return realName
}
// shouldSkip returns true if the field should be omitted (nil ptr, omitempty)
func (e *encoder) shouldSkip(container reflect.Value, realName string, val reflect.Value) bool {
// Skip nil pointers / interfaces
if (val.Kind() == reflect.Ptr || val.Kind() == reflect.Interface) && val.IsNil() {
return true
}
// Maps don't have tags, so we only skip nil values
if container.Kind() == reflect.Map {
return false
}
// Structs: check omitempty
field, _ := container.Type().FieldByName(realName)
tag := field.Tag.Get("toml")
if strings.Contains(tag, "omitempty") && isEmptyValue(val) {
return true
}
return false
}
// isTable determines if a value renders as [Table] / [[Array of Tables]].
// Slices must be homogeneous: mixing table and scalar elements is an error
// (parser accepts such arrays; encoder cannot represent them — documented
// limitation). Nil pointer elements are allowed in table slices and skipped
// at emission; nil interface elements are an error.
func (e *encoder) isTable(v reflect.Value) (bool, error) {
if v.Kind() == reflect.Interface {
v = v.Elem()
}
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
switch v.Kind() {
case reflect.Struct, reflect.Map:
return true, nil
case reflect.Slice, reflect.Array:
tables, scalars := 0, 0
for i := 0; i < v.Len(); i++ {
elem := v.Index(i)
if elem.Kind() == reflect.Interface {
if elem.IsNil() {
return false, fmt.Errorf("nil element at index %d", i)
}
elem = elem.Elem()
}
if elem.Kind() == reflect.Ptr {
if elem.IsNil() {
tables++ // nil pointers are only legal in [[table]] context (skipped)
continue
}
elem = elem.Elem()
}
switch elem.Kind() {
case reflect.Struct, reflect.Map:
tables++
default:
scalars++
}
}
if tables > 0 && scalars > 0 {
return false, fmt.Errorf("mixed table/scalar elements in array")
}
return tables > 0, nil
}
return false, nil
}
func isEmptyValue(v reflect.Value) bool {
switch v.Kind() {
case reflect.Array, reflect.Map, reflect.Slice, reflect.String:
return v.Len() == 0
case reflect.Bool:
return !v.Bool()
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return v.Int() == 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return v.Uint() == 0
case reflect.Float32, reflect.Float64:
return v.Float() == 0
case reflect.Interface, reflect.Ptr:
return v.IsNil()
}
return false
}
func (e *encoder) writeKey(s string) error {
if isBareKey(s) {
_, err := e.w.WriteString(s)
return err
}
e.encodeString(s)
return nil
}
func (e *encoder) encodeString(s string) {
e.w.WriteString("\"")
for _, r := range s {
switch r {
case '"':
e.w.WriteString(`\"`)
case '\\':
e.w.WriteString(`\\`)
case '\n':
e.w.WriteString(`\n`)
case '\r':
e.w.WriteString(`\r`)
case '\t':
e.w.WriteString(`\t`)
case '\b':
e.w.WriteString(`\b`)
case '\f':
e.w.WriteString(`\f`)
default:
if r < 0x20 || r == 0x7F {
e.w.WriteString(fmt.Sprintf(`\u%04X`, r))
} else {
e.w.WriteRune(r)
}
}
}
e.w.WriteString("\"")
}
// isBareKey determines if a string can be written as a bare key
// It follows the TOML spec (A-Za-z0-9_-) but also respects the provided Lexer's behavior
// If the Lexer would interpret the string as a Number or Boolean, it must be quoted because the Parser expects TokenIdent (or TokenString) for keys
func isBareKey(s string) bool {
if s == "" {
return false
}
// 1. Valid bare key characters: A-Za-z0-9_-
for _, r := range s {
if !((r >= 'A' && r <= 'Z') || (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') || r == '_' || r == '-') {
return false
}
}
// 2. Avoid collision with Booleans
// Lexer emits TokenBool for these, Parser expects TokenIdent.
if s == "true" || s == "false" {
return false
}
// 3. Avoid collision with Numbers
// The provided Lexer triggers number parsing if the token starts with a digit,
// or a '-' followed by a digit.
// Since the Lexer never produces TokenIdent for these cases (it produces TokenInteger/Float),
// and the Parser rejects numeric tokens as keys, we must quote them.
c0 := s[0]
if c0 >= '0' && c0 <= '9' {
return false
}
if c0 == '-' && len(s) > 1 {
c1 := s[1]
if c1 >= '0' && c1 <= '9' {
return false
}
}
return true
}
+203
View File
@@ -0,0 +1,203 @@
package toml
import (
"strings"
"testing"
)
func TestMarshal_Primitives(t *testing.T) {
tests := []struct {
name string
input map[string]any
expected string // partial match or exact
}{
{
name: "Scalars",
input: map[string]any{"str": "hello", "int": 42, "bool": true, "float": 3.14},
expected: `bool = true
float = 3.14
int = 42
str = "hello"`,
},
{
name: "Quoted Keys",
input: map[string]any{"123a": 1, "key.dot": 2, "true": 3},
expected: `"123a" = 1
"key.dot" = 2
"true" = 3`,
},
{
name: "Inline Arrays",
input: map[string]any{"arr": []int{1, 2, 3}},
expected: `arr = [1, 2, 3]`,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
b, err := Marshal(tc.input)
if err != nil {
t.Fatalf("Marshal failed: %v", err)
}
out := strings.TrimSpace(string(b))
if out != tc.expected {
t.Errorf("Mismatch:\nGot:\n%s\nWant:\n%s", out, tc.expected)
}
})
}
}
func TestMarshal_StructsAndNesting(t *testing.T) {
type Server struct {
IP string `toml:"ip"`
Port int `toml:"port"`
}
type Config struct {
Name string `toml:"name"`
Tags []string `toml:"tags"`
Servers []Server `toml:"servers"` // Array of tables
Meta map[string]string `toml:"meta"` // Table
}
input := Config{
Name: "Production",
Tags: []string{"web", "api"},
Servers: []Server{
{IP: "10.0.0.1", Port: 80},
{IP: "10.0.0.2", Port: 8080},
},
Meta: map[string]string{
"env": "prod",
"dc": "us-east",
},
}
b, err := Marshal(input)
if err != nil {
t.Fatalf("Marshal failed: %v", err)
}
out := string(b)
// Verify key order (determinism) and structure
// Scalars first: name, tags
if !strings.Contains(out, `name = "Production"`) {
t.Error("Missing name field")
}
if !strings.Contains(out, `tags = ["web", "api"]`) {
t.Error("Missing tags field")
}
// Tables later: meta
if !strings.Contains(out, `[meta]`) {
t.Error("Missing [meta] table")
}
if !strings.Contains(out, `env = "prod"`) {
t.Error("Missing env inside meta")
}
// Array of tables: servers
if !strings.Contains(out, `[[servers]]`) {
t.Error("Missing [[servers]] header")
}
if !strings.Contains(out, `ip = "10.0.0.1"`) {
t.Error("Missing server IP")
}
}
func TestMarshal_RoundTrip(t *testing.T) {
// Complex input covering most features
input := map[string]any{
"title": "Symmetry Test",
"owner": map[string]any{
"name": "Tom",
"dob": "1979-05-27T07:32:00Z", // String because date support is limited
},
"database": map[string]any{
"server": "192.168.1.1",
"ports": []any{8001, 8001, 8002},
"connection_max": 5000,
"enabled": true,
},
"servers": []map[string]any{
{"ip": "10.0.0.1", "role": "frontend"},
{"ip": "10.0.0.2", "role": "backend"},
},
"quoted-keys": map[string]any{
// "1234": "val" would fail because Parser explicitly forbids keys that validly Atoi()
"1234a": "alphanumeric starting with digit", // Should be quoted in output, accepted by Parser
"a-b": "bare key", // Should not be quoted
"true": "bool key", // Should be quoted
},
}
// 1. Marshal
data, err := Marshal(input)
if err != nil {
t.Fatalf("Marshal failed: %v", err)
}
// 2. Unmarshal back
var output map[string]any
if err := Unmarshal(data, &output); err != nil {
t.Fatalf("Unmarshal failed on generated output: %v\nOutput:\n%s", err, string(data))
}
// 3. Compare
if input["title"] != output["title"] {
t.Errorf("Title mismatch: %v != %v", input["title"], output["title"])
}
dbIn := input["database"].(map[string]any)
dbOut := output["database"].(map[string]any)
if dbIn["server"] != dbOut["server"] {
t.Error("Database server mismatch")
}
// Check quoted keys
qk := output["quoted-keys"].(map[string]any)
if qk["1234a"] != "alphanumeric starting with digit" {
t.Error("Failed to round-trip numeric-like key '1234a'")
}
if qk["true"] != "bool key" {
t.Error("Failed to round-trip boolean key 'true'")
}
}
func TestMarshal_Omitempty(t *testing.T) {
type Config struct {
Visible string `toml:"visible"`
Hidden string `toml:"hidden,omitempty"`
Zero int `toml:"zero,omitempty"`
}
cfg := Config{Visible: "here"}
b, err := Marshal(cfg)
if err != nil {
t.Fatalf("Marshal failed: %v", err)
}
out := string(b)
if !strings.Contains(out, `visible = "here"`) {
t.Error("Visible field missing")
}
if strings.Contains(out, "hidden") {
t.Error("Hidden field present but should be omitted")
}
if strings.Contains(out, "zero") {
t.Error("Zero field present but should be omitted")
}
}
func TestMarshal_SkipNil(t *testing.T) {
type Config struct {
Ptr *int `toml:"ptr"`
}
cfg := Config{Ptr: nil}
b, err := Marshal(cfg)
if err != nil {
t.Fatalf("Marshal failed: %v", err)
}
if len(b) > 0 {
t.Errorf("Expected empty output for nil pointer, got: %s", string(b))
}
}
+87
View File
@@ -0,0 +1,87 @@
package toml
import (
"reflect"
"testing"
"unicode/utf8"
)
func FuzzParse(f *testing.F) {
for _, s := range []string{
"",
`key = "value"`,
"[table]\nk = 1",
"[[arr]]\nx = 1.5\n[[arr]]\nx = 2e3",
"t = { a = 1, b = { c = [1, 2] } }",
`s = "esc \u00E9 \t"`,
"n = -0x10\nb = 0b101\no = 0o17",
} {
f.Add([]byte(s))
}
f.Fuzz(func(t *testing.T, data []byte) {
p := NewParser(data)
_, _ = p.Parse() // property: no panic, no hang
})
}
// normalize collapses the parser's dual slice representations
// ([]map[string]any for [[headers]], []any for inline arrays) so the
// fixpoint compares values, not construction syntax.
func normalize(v any) any {
switch t := v.(type) {
case map[string]any:
for k, vv := range t {
t[k] = normalize(vv)
}
return t
case []map[string]any:
out := make([]any, len(t))
for i, m := range t {
out[i] = normalize(m)
}
return out
case []any:
for i, vv := range t {
t[i] = normalize(vv)
}
return t
}
return v
}
func FuzzRoundTrip(f *testing.F) {
for _, s := range []string{
"a = 1\nb = 2.5\nc = true\nd = \"x\"",
"[t]\nk = [1, 2, 3]",
"[[s]]\nn = \"a\"\n[[s]]\nn = \"b\"",
`m = { x = 1, y = { z = "q" } }`,
`u = "日本語 \u00E9"`,
"big = 1e100\nsmall = 1e-100",
} {
f.Add([]byte(s))
}
f.Fuzz(func(t *testing.T, data []byte) {
// Encoder normalizes invalid UTF-8 to U+FFFD (WriteRune); byte-exact
// round trip is only a property of valid input.
if !utf8.Valid(data) {
t.Skip()
}
m1, err := NewParser(data).Parse()
if err != nil {
t.Skip()
}
out, err := Marshal(m1)
if err != nil {
// Parser accepts values the encoder cannot represent
// (mixed table/scalar arrays). Documented asymmetry.
t.Skip()
}
m2, err := NewParser(out).Parse()
if err != nil {
t.Fatalf("emitted TOML failed to re-parse: %v\nemitted:\n%s", err, out)
}
if !reflect.DeepEqual(normalize(m1), normalize(m2)) {
t.Fatalf("fixpoint violation:\nm1: %#v\nm2: %#v\nemitted:\n%s", m1, m2, out)
}
})
}
+3
View File
@@ -0,0 +1,3 @@
module github.com/lixenwraith/toml
go 1.24.0
+430
View File
@@ -0,0 +1,430 @@
package toml
import (
"fmt"
"unicode/utf8"
)
type Lexer struct {
input []byte
pos int
line int
col int
width int
}
func NewLexer(input []byte) *Lexer {
return &Lexer{
input: input,
line: 1,
col: 0,
}
}
func (l *Lexer) NextToken() Token {
l.skipWhitespace()
if l.pos >= len(l.input) {
return l.newToken(TokenEOF, "")
}
ch := l.peek()
if ch == '\n' {
l.advance()
return l.newToken(TokenNewline, "\n")
}
if ch == '#' {
return l.readComment()
}
switch ch {
case '=':
l.advance()
return l.newToken(TokenEqual, "=")
case '.':
l.advance()
return l.newToken(TokenDot, ".")
case ',':
l.advance()
return l.newToken(TokenComma, ",")
case '[':
l.advance()
return l.newToken(TokenLBracket, "[")
case ']':
l.advance()
return l.newToken(TokenRBracket, "]")
case '{':
l.advance()
return l.newToken(TokenLBrace, "{")
case '}':
l.advance()
return l.newToken(TokenRBrace, "}")
case '"':
return l.readString()
}
// Number: digit or sign+digit
if isDigit(ch) {
return l.readNumber()
}
if ch == '+' {
if isDigit(l.peekAt(1)) {
return l.readNumber()
}
// Lone + is invalid TOML
l.advance()
return l.newToken(TokenError, "unexpected character: +")
}
if ch == '-' {
if isDigit(l.peekAt(1)) {
return l.readNumber()
}
// Bare key can start with hyphen
return l.readIdent()
}
// Identifier: alpha or underscore
if isAlpha(ch) || ch == '_' {
return l.readIdent()
}
l.advance()
return l.newToken(TokenError, fmt.Sprintf("unexpected character: %c", ch))
}
func (l *Lexer) readNumber() Token {
start := l.pos
startCol := l.col
// Optional sign
if l.peek() == '+' || l.peek() == '-' {
l.advance()
}
// Check radix prefix
if l.peek() == '0' {
next := l.peekAt(1)
switch next {
case 'x', 'X':
l.advance() // '0'
l.advance() // 'x'
return l.readHex(start, startCol)
case 'o', 'O':
l.advance()
l.advance()
return l.readOctal(start, startCol)
case 'b', 'B':
l.advance()
l.advance()
return l.readBinary(start, startCol)
}
}
// Integer part
for isDigit(l.peek()) {
l.advance()
}
isFloat := false
// Fractional part: '.' followed by digit
if l.peek() == '.' && isDigit(l.peekAt(1)) {
isFloat = true
l.advance() // consume '.'
for isDigit(l.peek()) {
l.advance()
}
// Multi-dot: another '.' followed by digit is fatal
if l.peek() == '.' && isDigit(l.peekAt(1)) {
return Token{Type: TokenError, Literal: "invalid number: multiple decimal points", Line: l.line, Col: startCol}
}
}
// Exponent: e/E followed by optional sign and digits
if l.peek() == 'e' || l.peek() == 'E' {
next := l.peekAt(1)
validExp := isDigit(next) || ((next == '+' || next == '-') && isDigit(l.peekAt(2)))
if validExp {
isFloat = true
l.advance() // 'e'
if l.peek() == '+' || l.peek() == '-' {
l.advance()
}
for isDigit(l.peek()) {
l.advance()
}
}
}
lit := string(l.input[start:l.pos])
// Validate float format: 1.e2 is invalid (no digit after dot before exponent)
if isFloat {
if err := validateFloat(lit); err != nil {
return Token{Type: TokenError, Literal: err.Error(), Line: l.line, Col: startCol}
}
return Token{Type: TokenFloat, Literal: lit, Line: l.line, Col: startCol}
}
return Token{Type: TokenInteger, Literal: lit, Line: l.line, Col: startCol}
}
func validateFloat(lit string) error {
// Find dot position (if any)
dotIdx := -1
expIdx := -1
for i := 0; i < len(lit); i++ {
if lit[i] == '.' {
dotIdx = i
}
if lit[i] == 'e' || lit[i] == 'E' {
expIdx = i
break
}
}
if dotIdx >= 0 {
// Check digit before dot (ignoring sign)
start := 0
if lit[0] == '+' || lit[0] == '-' {
start = 1
}
if dotIdx == start {
return fmt.Errorf("invalid float %q: no digit before decimal point", lit)
}
// Check digit after dot
afterDot := dotIdx + 1
endFrac := len(lit)
if expIdx > 0 {
endFrac = expIdx
}
if afterDot >= endFrac {
return fmt.Errorf("invalid float %q: no digit after decimal point", lit)
}
}
return nil
}
func (l *Lexer) readHex(start, startCol int) Token {
if !isHexDigit(l.peek()) {
return Token{Type: TokenError, Literal: "invalid hex: no digits after prefix", Line: l.line, Col: startCol}
}
for isHexDigit(l.peek()) {
l.advance()
}
return Token{Type: TokenInteger, Literal: string(l.input[start:l.pos]), Line: l.line, Col: startCol}
}
func (l *Lexer) readOctal(start, startCol int) Token {
if !isOctalDigit(l.peek()) {
return Token{Type: TokenError, Literal: "invalid octal: no digits after prefix", Line: l.line, Col: startCol}
}
for isOctalDigit(l.peek()) {
l.advance()
}
return Token{Type: TokenInteger, Literal: string(l.input[start:l.pos]), Line: l.line, Col: startCol}
}
func (l *Lexer) readBinary(start, startCol int) Token {
if !isBinaryDigit(l.peek()) {
return Token{Type: TokenError, Literal: "invalid binary: no digits after prefix", Line: l.line, Col: startCol}
}
for isBinaryDigit(l.peek()) {
l.advance()
}
return Token{Type: TokenInteger, Literal: string(l.input[start:l.pos]), Line: l.line, Col: startCol}
}
func (l *Lexer) readIdent() Token {
start := l.pos
for l.pos < len(l.input) {
ch := l.peek()
if isAlpha(ch) || isDigit(ch) || ch == '_' || ch == '-' {
l.advance()
} else {
break
}
}
lit := string(l.input[start:l.pos])
if lit == "true" || lit == "false" {
return l.newToken(TokenBool, lit)
}
return l.newToken(TokenIdent, lit)
}
func (l *Lexer) newToken(typ TokenType, literal string) Token {
col := l.col - len(literal)
if col < 0 {
col = 0
}
return Token{Type: typ, Literal: literal, Line: l.line, Col: col}
}
func (l *Lexer) advance() rune {
if l.pos >= len(l.input) {
l.width = 0
return 0
}
r, w := utf8.DecodeRune(l.input[l.pos:])
l.width = w
l.pos += w
if r == '\n' {
l.line++
l.col = 0
} else {
l.col++
}
return r
}
func (l *Lexer) peek() rune {
if l.pos >= len(l.input) {
return 0
}
r, _ := utf8.DecodeRune(l.input[l.pos:])
return r
}
func (l *Lexer) peekAt(n int) rune {
pos := l.pos
for i := 0; i < n && pos < len(l.input); i++ {
_, w := utf8.DecodeRune(l.input[pos:])
pos += w
}
if pos >= len(l.input) {
return 0
}
r, _ := utf8.DecodeRune(l.input[pos:])
return r
}
func (l *Lexer) skipWhitespace() {
for l.pos < len(l.input) {
ch := l.peek()
if ch == ' ' || ch == '\t' || ch == '\r' {
l.advance()
} else {
break
}
}
}
func (l *Lexer) readComment() Token {
l.advance() // '#'
start := l.pos
for l.pos < len(l.input) && l.peek() != '\n' {
l.advance()
}
return l.newToken(TokenComment, string(l.input[start:l.pos]))
}
func (l *Lexer) readString() Token {
l.advance() // opening quote
var result []byte
escaped := false
for l.pos < len(l.input) {
ch := l.peek()
if ch == '\n' {
return l.newToken(TokenError, "unterminated string: newline in basic string")
}
if ch == '"' && !escaped {
l.advance()
return l.newToken(TokenString, string(result))
}
if ch == '\\' && !escaped {
escaped = true
l.advance()
continue
}
if escaped {
switch ch {
case '"':
result = append(result, '"')
case '\\':
result = append(result, '\\')
case 'n':
result = append(result, '\n')
case 't':
result = append(result, '\t')
case 'r':
result = append(result, '\r')
case 'u', 'U':
n := 4
if ch == 'U' {
n = 8
}
l.advance() // consume 'u'/'U'
var code rune
for j := 0; j < n; j++ {
d, ok := hexDigitVal(l.peek())
if !ok {
return l.newToken(TokenError, "invalid unicode escape: expected hex digit")
}
code = code<<4 | rune(d)
l.advance()
}
// ValidRune rejects surrogates and > U+10FFFF (spec: scalar values only).
// \UFFFFFFFF overflows rune's sign bit -> negative -> rejected here too.
if !utf8.ValidRune(code) {
return l.newToken(TokenError, fmt.Sprintf("invalid unicode escape: U+%X is not a scalar value", code))
}
var buf [utf8.UTFMax]byte
w := utf8.EncodeRune(buf[:], code)
result = append(result, buf[:w]...)
escaped = false
continue // hex digits already consumed; skip trailing l.advance()
default:
// Unknown escape: preserve backslash and full rune
result = append(result, '\\')
var buf [utf8.UTFMax]byte
n := utf8.EncodeRune(buf[:], ch)
result = append(result, buf[:n]...)
}
escaped = false
} else {
// Get actual width of current character, don't use stale l.width
_, w := utf8.DecodeRune(l.input[l.pos:])
result = append(result, l.input[l.pos:l.pos+w]...)
}
l.advance()
}
return l.newToken(TokenError, "unterminated string")
}
func isDigit(r rune) bool {
return r >= '0' && r <= '9'
}
func isAlpha(r rune) bool {
return (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z')
}
func isHexDigit(r rune) bool {
return isDigit(r) || (r >= 'a' && r <= 'f') || (r >= 'A' && r <= 'F')
}
func hexDigitVal(r rune) (int, bool) {
switch {
case r >= '0' && r <= '9':
return int(r - '0'), true
case r >= 'a' && r <= 'f':
return int(r-'a') + 10, true
case r >= 'A' && r <= 'F':
return int(r-'A') + 10, true
}
return 0, false
}
func isOctalDigit(r rune) bool {
return r >= '0' && r <= '7'
}
func isBinaryDigit(r rune) bool {
return r == '0' || r == '1'
}
+517
View File
@@ -0,0 +1,517 @@
package toml
import (
"strings"
"testing"
)
func TestDecode_UnexportedFieldPanic(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("Recovered from panic: %v. Logic should skip unexported fields.", r)
}
}()
data := map[string]any{"secret": "hacker"}
type Security struct {
secret string
Public string `toml:"secret"`
}
var s Security
_ = Decode(data, &s)
}
func TestLexer_InvalidNumbers(t *testing.T) {
tests := []struct {
input string
wantErr bool
}{
{"1.0a", true}, // Not valid TOML document structure
{"1.-0", true}, // Not valid TOML document structure
{"0xG1", true}, // Invalid hex digit
{"+", true}, // Lone + is invalid TOML
{"[1.2.3]", true}, // Multi-dot in numeric context
}
for _, tc := range tests {
p := NewParser([]byte(tc.input))
_, err := p.Parse()
if tc.wantErr && err == nil {
t.Errorf("Input %q should have failed parsing", tc.input)
}
if !tc.wantErr && err != nil {
t.Errorf("Input %q unexpected error: %v", tc.input, err)
}
}
}
func TestDecode_DeepPointers(t *testing.T) {
data := map[string]any{"val": 42}
type T struct {
Val ******int `toml:"val"`
}
var tgt T
if err := Decode(data, &tgt); err != nil {
t.Fatalf("Deep pointer decode failed: %v", err)
}
if ******tgt.Val != 42 {
t.Errorf("Expected 42, got %d", ******tgt.Val)
}
}
func TestDecode_LargeIntPrecision(t *testing.T) {
largeVal := int64(4611686018427387905)
data := map[string]any{"id": int(largeVal)}
type T struct {
ID int64 `toml:"id"`
}
var tgt T
_ = Decode(data, &tgt)
if tgt.ID != largeVal {
t.Errorf("Precision loss detected: got %d, want %d", tgt.ID, largeVal)
}
}
func TestParser_NumericKeyRejection(t *testing.T) {
inputs := [][]byte{
[]byte(`123 = "value"`),
[]byte(`[123]`),
[]byte(`[a.123.b]`),
}
for _, in := range inputs {
p := NewParser(in)
if _, err := p.Parse(); err == nil {
t.Errorf("Parser should have rejected numeric key in: %s", string(in))
}
}
}
func TestPanic_LexerInfinity(t *testing.T) {
input := []byte("key = \"\x00\xff\"\n[table\x00]")
l := NewLexer(input)
for i := 0; i < 100; i++ {
tok := l.NextToken()
if tok.Type == TokenEOF {
return
}
}
t.Error("Lexer likely stuck in infinite loop on invalid input")
}
func TestPanic_DeepNesting(t *testing.T) {
depth := 1000
input := strings.Repeat("a.", depth) + "b = 1"
p := NewParser([]byte(input))
_, err := p.Parse()
if err != nil && !strings.Contains(err.Error(), "key path conflict") {
t.Logf("Caught expected deep nesting error: %v", err)
}
}
func TestBreak_TableRedefinition(t *testing.T) {
input := []byte(`
anchor = 1
[anchor]
sub = 2
`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Parser failed to catch redefinition of a value as a table")
}
}
func TestBreak_MalformedScientificNotation(t *testing.T) {
tests := []string{
"val = 1e",
"val = 1e+",
"val = .5",
"val = 1.e2",
}
for _, tc := range tests {
p := NewParser([]byte(tc))
_, err := p.Parse()
if err == nil {
t.Errorf("Should have failed to parse malformed float: %s", tc)
}
}
}
func TestBreak_SliceTypeMismatch(t *testing.T) {
data := map[string]any{
"list": []any{1, "string", 3},
}
type Target struct {
List []int `toml:"list"`
}
var tgt Target
err := Decode(data, &tgt)
if err == nil {
t.Error("Decoder should have failed converting string to int inside slice")
}
}
func TestBreak_InvalidDottedKeyInInlineTable(t *testing.T) {
input := []byte(`config = { valid.123 = "fail" }`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Parser allowed numeric segment in dotted inline table key")
}
}
func TestPanic_NilInterfaceAssignment(t *testing.T) {
defer func() {
if r := recover(); r != nil {
t.Errorf("Panic during nil interface decoding: %v", r)
}
}()
var target any
data := map[string]any{"a": 1}
_ = Decode(data, &target)
}
func TestStructural_NestedReentry(t *testing.T) {
input := []byte(`
[a.b.c]
depth = 3
[a]
root_val = 1
[a.b]
mid_val = 2
`)
p := NewParser(input)
res, err := p.Parse()
if err != nil {
t.Fatalf("Valid nested reentry failed: %v", err)
}
a := res["a"].(map[string]any)
if a["root_val"] != 1 {
t.Errorf("Missing root_val: %v", a["root_val"])
}
b := a["b"].(map[string]any)
if b["mid_val"] != 2 {
t.Errorf("Missing mid_val: %v", b["mid_val"])
}
}
func TestBreak_KeyCollisionDotted(t *testing.T) {
input := []byte(`
a.b = 1
[a.b]
c = 2
`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Should have failed: redefining scalar a.b as a table")
}
}
func TestBreak_IntegerOverflow(t *testing.T) {
input := []byte(`val = 9223372036854775808`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Parser should have errored on int64 overflow")
}
}
func TestBreak_ArrayTableShadowing(t *testing.T) {
input := []byte(`
[conflict]
sub = 1
[[conflict]]
sub = 2
`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Should have failed: [conflict] followed by [[conflict]]")
}
}
func TestBreak_RecursiveDecoder(t *testing.T) {
type Recursive struct {
Next *Recursive `toml:"next"`
}
data := map[string]any{
"next": map[string]any{
"next": map[string]any{
"next": map[string]any{},
},
},
}
var target Recursive
err := Decode(data, &target)
if err != nil {
t.Fatalf("Recursive decode failed: %v", err)
}
if target.Next.Next.Next == nil {
t.Error("Recursive decoding depth mismatch")
}
}
func TestBreak_DottedKeyConflictWithTable(t *testing.T) {
input := []byte(`
[a]
b.c = 1
[a.b]
c = 2
`)
p := NewParser(input)
_, err := p.Parse()
if err == nil {
t.Error("Should have failed: duplicate definition of a.b.c")
}
}
func TestLexer_CommentEdgeCases(t *testing.T) {
input := []byte(`
key = "value # not a comment" # this is a comment
# Empty line with comment
# indented comment
[table] # comment after table
`)
p := NewParser(input)
res, err := p.Parse()
if err != nil {
t.Fatalf("Lexer failed on valid comments: %v", err)
}
if res["key"] != "value # not a comment" {
t.Errorf("Comment in string was incorrectly truncated: %v", res["key"])
}
}
func TestLexer_StrictNumericValidation(t *testing.T) {
tests := []struct {
name string
input string
expected []TokenType
}{
{
"Multiple dots error",
"1.1.1",
[]TokenType{TokenError},
},
{
"Octal is valid",
"val = 0123",
[]TokenType{TokenIdent, TokenEqual, TokenInteger, TokenEOF},
},
{
"Negative with leading zero valid",
"val = -01",
[]TokenType{TokenIdent, TokenEqual, TokenInteger, TokenEOF},
},
{
"Float then dot then int",
"1e1.5",
[]TokenType{TokenFloat, TokenDot, TokenInteger, TokenEOF},
},
{
"Float then ident",
"1e1e1",
[]TokenType{TokenFloat, TokenIdent, TokenEOF},
},
{
"Float then ident with dots",
"1.00a00",
[]TokenType{TokenFloat, TokenIdent, TokenEOF},
},
{
"Incomplete exponent error",
"val = 1e+",
[]TokenType{TokenIdent, TokenEqual, TokenInteger, TokenIdent, TokenError},
},
{
"Zero valid",
"val = 0",
[]TokenType{TokenIdent, TokenEqual, TokenInteger, TokenEOF},
},
{
"Negative zero valid",
"val = -0",
[]TokenType{TokenIdent, TokenEqual, TokenInteger, TokenEOF},
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
l := NewLexer([]byte(tc.input))
var got []TokenType
for {
tok := l.NextToken()
got = append(got, tok.Type)
if tok.Type == TokenEOF || tok.Type == TokenError {
break
}
}
if len(got) != len(tc.expected) {
t.Errorf("[%s] token count: got %d %v, want %d %v", tc.input, len(got), got, len(tc.expected), tc.expected)
return
}
for i, exp := range tc.expected {
if got[i] != exp {
t.Errorf("[%s] token[%d]: got %v, want %v", tc.input, i, got[i], exp)
}
}
})
}
}
func TestParser_KeyPathDeepExhaustion(t *testing.T) {
input := []byte(`
a.b.c.d.e = 1
a.b.c.f = 2
[a.b.c]
g = 3
[a.b.c.d]
h = 4
[a.b]
i = 5
`)
p := NewParser(input)
res, err := p.Parse()
if err != nil {
t.Fatalf("Failed on complex but valid nested reentry: %v", err)
}
a := res["a"].(map[string]any)
b := a["b"].(map[string]any)
if b["i"] != 5 {
t.Errorf("Value 'i' lost in table reentry. Got %v", b["i"])
}
if _, ok := b["c"].(map[string]any); !ok {
t.Errorf("Sub-map 'c' lost during parent reentry")
}
}
func TestParser_FloatParsingErrors(t *testing.T) {
tests := []string{
"f = .5",
"f = 1.",
}
for _, tc := range tests {
p := NewParser([]byte(tc))
_, err := p.Parse()
if err == nil {
t.Errorf("Should have failed to parse: %s", tc)
}
}
}
func TestLexer_HexWithE(t *testing.T) {
// 0xDEAD must be Integer, not misclassified as Float due to 'E'
input := "val = 0xDEAD"
l := NewLexer([]byte(input))
tok := l.NextToken() // val
if tok.Type != TokenIdent {
t.Errorf("Expected Ident, got %v", tok.Type)
}
tok = l.NextToken() // =
if tok.Type != TokenEqual {
t.Errorf("Expected Equal, got %v", tok.Type)
}
tok = l.NextToken() // 0xDEAD
if tok.Type != TokenInteger {
t.Errorf("Expected Integer for hex, got %v (%s)", tok.Type, tok.Literal)
}
if tok.Literal != "0xDEAD" {
t.Errorf("Literal mismatch: %q", tok.Literal)
}
}
func TestLexer_IPAddressAndVersion(t *testing.T) {
// IP-like or semver must error on multi-dot
input := "version = 1.2.3"
l := NewLexer([]byte(input))
tok := l.NextToken() // version
if tok.Type != TokenIdent {
t.Errorf("Expected Ident, got %v", tok.Type)
}
tok = l.NextToken() // =
if tok.Type != TokenEqual {
t.Errorf("Expected Equal, got %v", tok.Type)
}
tok = l.NextToken() // 1.2.3 should error
if tok.Type != TokenError {
t.Errorf("Expected Error for multi-dot, got %v (%s)", tok.Type, tok.Literal)
}
}
func TestParser_StrictNoNumericKeys(t *testing.T) {
tests := []struct {
input string
name string
}{
{`123 = "val"`, "Bare integer key"},
{`[123]`, "Integer table header"},
{`["456"]`, "Quoted integer key"},
{`a.1.b = "val"`, "Numeric segment in dotted key"},
}
for _, tc := range tests {
p := NewParser([]byte(tc.input))
_, err := p.Parse()
if err == nil {
t.Errorf("Failed %s: should have rejected numeric key", tc.name)
}
}
}
func TestLexer_AmbiguousNumericDotted(t *testing.T) {
tests := []struct {
input string
expected []TokenType
}{
{"1.a", []TokenType{TokenInteger, TokenDot, TokenIdent, TokenEOF}},
{"1.0.0", []TokenType{TokenError}}, // Multi-dot
{"1e1.5", []TokenType{TokenFloat, TokenDot, TokenInteger, TokenEOF}},
}
for _, tc := range tests {
l := NewLexer([]byte(tc.input))
var got []TokenType
for {
tok := l.NextToken()
got = append(got, tok.Type)
if tok.Type == TokenEOF || tok.Type == TokenError {
break
}
}
if len(got) != len(tc.expected) {
t.Errorf("%s: got %v, want %v", tc.input, got, tc.expected)
continue
}
for i, exp := range tc.expected {
if got[i] != exp {
t.Errorf("%s[%d]: got %v, want %v", tc.input, i, got[i], exp)
}
}
}
}
func TestParser_KeyValueContext(t *testing.T) {
input := `key = 1.1`
p := NewParser([]byte(input))
_, err := p.Parse()
if err != nil {
t.Errorf("Valid float value failed: %v", err)
}
input2 := `1.1 = "value"`
p2 := NewParser([]byte(input2))
_, err = p2.Parse()
if err == nil {
t.Error("Float key should have been rejected")
}
}
+435
View File
@@ -0,0 +1,435 @@
package toml
import (
"fmt"
"reflect"
"strconv"
)
// Enforce 64-bit platform; fails compilation on 32-bit targets where int(int64) in parseInteger would truncate
const _ uint = 1<<63 - 1
// Parser parses TOML tokens into a map[string]any
type Parser struct {
root map[string]any
lexer *Lexer
current any // Pointer to the current map or slice of maps being populated (scope)
// Identity set of inline-table maps (immutable per TOML spec)
frozen map[uintptr]bool
curToken Token
peekToken Token
// Value nesting depth (arrays / inline tables)
depth int
}
// Recursion bound for parseValue -> parseArray/parseInlineTable
const maxValueDepth = 1000
func NewParser(input []byte) *Parser {
l := NewLexer(input)
p := &Parser{
lexer: l,
root: make(map[string]any),
frozen: make(map[uintptr]bool),
}
p.nextToken()
p.nextToken()
p.current = p.root
return p
}
func (p *Parser) nextToken() {
p.curToken = p.peekToken
p.peekToken = p.lexer.NextToken()
// Skip comments automatically
for p.peekToken.Type == TokenComment {
p.peekToken = p.lexer.NextToken()
}
}
func (p *Parser) Parse() (map[string]any, error) {
for p.curToken.Type != TokenEOF {
if p.curToken.Type == TokenNewline {
p.nextToken()
continue
}
if err := p.parseStatement(); err != nil {
return nil, err
}
}
return p.root, nil
}
func (p *Parser) parseStatement() error {
switch p.curToken.Type {
case TokenLBracket:
// Table Definition: [table] or [[array.table]]
return p.parseTableDeclaration()
case TokenIdent, TokenString:
// Key-Value Pair: key = value
return p.parseKeyValuePair(p.current)
case TokenError:
return fmt.Errorf("lexing error line %d: %s", p.curToken.Line, p.curToken.Literal)
default:
return fmt.Errorf("unexpected token line %d: %s", p.curToken.Line, p.curToken.String())
}
}
// parseTableDeclaration handles [key] and [[key]]
func (p *Parser) parseTableDeclaration() error {
isArray := false
if p.peekToken.Type == TokenLBracket {
// It is [[ ...
p.nextToken() // consume first [
isArray = true
}
p.nextToken() // consume [
// Parse Key (dotted)
keys, err := p.parseKeyParts()
if err != nil {
return err
}
if isArray {
if p.curToken.Type != TokenRBracket {
return fmt.Errorf("expected closing bracket for array table at line %d", p.curToken.Line)
}
p.nextToken() // consume first ]
}
if p.curToken.Type != TokenRBracket {
return fmt.Errorf("expected closing bracket for table at line %d", p.curToken.Line)
}
p.nextToken() // consume final ]
// Define scope
return p.setTableScope(keys, isArray)
}
// setTableScope navigates/creates the map structure and sets p.current
func (p *Parser) setTableScope(keys []string, isArrayOfTables bool) error {
// Table declarations always start from root
var ptr any = p.root
for i, key := range keys {
isLast := i == len(keys)-1
currentMap, ok := ptr.(map[string]any)
if !ok {
return fmt.Errorf("key path conflict: %s is not a map", key)
}
if isLast {
if isArrayOfTables {
// [[a.b]] -> Ensure 'b' is a slice of maps, append new map, set cursor to it
var slice []map[string]any
if val, exists := currentMap[key]; exists {
if s, ok := val.([]map[string]any); ok {
slice = s
} else {
return fmt.Errorf("key conflict: %s is not an array of tables", key)
}
} else {
slice = make([]map[string]any, 0)
}
newMap := make(map[string]any)
slice = append(slice, newMap)
currentMap[key] = slice
p.current = newMap
} else {
// [a.b] -> Ensure 'b' is a map, set cursor to it
var targetMap map[string]any
if val, exists := currentMap[key]; exists {
if m, ok := val.(map[string]any); ok {
// Inline tables cannot be reopened
if p.frozen[reflect.ValueOf(m).Pointer()] {
return fmt.Errorf("cannot extend inline table %q at line %d", key, p.curToken.Line)
}
targetMap = m
} else {
return fmt.Errorf("key conflict: %s is not a table", key)
}
} else {
targetMap = make(map[string]any)
currentMap[key] = targetMap
}
p.current = targetMap
}
} else {
// Intermediate key -> ensure map exists and traverse.
// Traversal through an existing [[array]] descends into its last element.
if val, exists := currentMap[key]; exists {
if m, ok := val.(map[string]any); ok {
// Inline tables cannot be extended via sub-tables
if p.frozen[reflect.ValueOf(m).Pointer()] {
return fmt.Errorf("cannot extend inline table %q at line %d", key, p.curToken.Line)
}
ptr = m
} else if slice, ok := val.([]map[string]any); ok {
if len(slice) == 0 {
return fmt.Errorf("cannot traverse empty array table %s", key)
}
ptr = slice[len(slice)-1]
} else {
return fmt.Errorf("intermediate key %s is not a map", key)
}
} else {
newMap := make(map[string]any)
currentMap[key] = newMap
ptr = newMap
}
}
}
return nil
}
func (p *Parser) parseKeyValuePair(scope any) error {
// Parse Key (dotted allowed: a.b.c = 1)
keys, err := p.parseKeyParts()
if err != nil {
return err
}
if p.curToken.Type != TokenEqual {
return fmt.Errorf("expected '=' after key at line %d, got %s", p.curToken.Line, p.curToken.String())
}
p.nextToken() // consume =
val, err := p.parseValue()
if err != nil {
return err
}
// Assign value to scope
return p.assignValue(scope, keys, val)
}
func (p *Parser) assignValue(scope any, keys []string, val any) error {
ptr := scope
// If scope is map, easy. If scope is not map, error.
currentMap, ok := ptr.(map[string]any)
if !ok {
return fmt.Errorf("scope is not a map")
}
for i, key := range keys {
if i == len(keys)-1 {
// Final key, assign value
if _, exists := currentMap[key]; exists {
return fmt.Errorf("duplicate key %s at line %d", key, p.curToken.Line)
}
currentMap[key] = val
} else {
// Intermediate, ensure map
if existing, exists := currentMap[key]; exists {
if m, ok := existing.(map[string]any); ok {
if p.frozen[reflect.ValueOf(m).Pointer()] {
return fmt.Errorf("cannot extend inline table %q at line %d", key, p.curToken.Line)
}
currentMap = m
} else {
return fmt.Errorf("intermediate key %s is not a map", key)
}
} else {
newMap := make(map[string]any)
currentMap[key] = newMap
currentMap = newMap
}
}
}
return nil
}
func (p *Parser) parseKeyParts() ([]string, error) {
var keys []string
for {
// Rule: Tokens identified as Numbers are forbidden as keys
if p.curToken.Type == TokenInteger || p.curToken.Type == TokenFloat {
return nil, fmt.Errorf("numeric keys are forbidden: %q", p.curToken.Literal)
}
if p.curToken.Type == TokenString {
// Rule: Even quoted strings shouldn't be pure numbers per instruction
if _, err := strconv.Atoi(p.curToken.Literal); err == nil {
return nil, fmt.Errorf("numeric string keys are forbidden: %q", p.curToken.Literal)
}
}
if p.curToken.Type != TokenIdent && p.curToken.Type != TokenString {
return nil, fmt.Errorf("expected key, got %s", p.curToken.String())
}
keys = append(keys, p.curToken.Literal)
p.nextToken()
if p.curToken.Type == TokenDot {
p.nextToken()
continue
}
break
}
return keys, nil
}
func (p *Parser) parseValue() (any, error) {
// Guard unbounded recursion on nested
p.depth++
defer func() { p.depth-- }()
if p.depth > maxValueDepth {
return nil, fmt.Errorf("value nesting exceeds %d at line %d", maxValueDepth, p.curToken.Line)
}
switch p.curToken.Type {
case TokenString:
val := p.curToken.Literal
p.nextToken()
return val, nil
case TokenInteger:
val, err := p.parseInteger(p.curToken.Literal)
if err != nil {
return nil, fmt.Errorf("invalid integer %q at line %d: %w", p.curToken.Literal, p.curToken.Line, err)
}
p.nextToken()
return val, nil
case TokenFloat:
val, err := strconv.ParseFloat(p.curToken.Literal, 64)
if err != nil {
return nil, fmt.Errorf("invalid float %q at line %d: %w", p.curToken.Literal, p.curToken.Line, err)
}
p.nextToken()
return val, nil
case TokenBool:
val := p.curToken.Literal == "true"
p.nextToken()
return val, nil
case TokenLBracket:
return p.parseArray()
case TokenLBrace:
return p.parseInlineTable()
}
return nil, fmt.Errorf("unexpected value token %s at line %d", p.curToken.String(), p.curToken.Line)
}
func (p *Parser) parseInteger(lit string) (int, error) {
// Handle optional leading sign
negative := false
numLit := lit
if len(numLit) > 0 && (numLit[0] == '+' || numLit[0] == '-') {
negative = numLit[0] == '-'
numLit = numLit[1:]
}
var val int64
var err error
if len(numLit) > 2 && numLit[0] == '0' {
switch numLit[1] {
case 'x', 'X':
val, err = strconv.ParseInt(numLit[2:], 16, 64)
case 'o', 'O':
val, err = strconv.ParseInt(numLit[2:], 8, 64)
case 'b', 'B':
val, err = strconv.ParseInt(numLit[2:], 2, 64)
default:
val, err = strconv.ParseInt(lit, 10, 64)
return int(val), err
}
if err != nil {
return 0, err
}
if negative {
val = -val
}
return int(val), nil
}
val, err = strconv.ParseInt(lit, 10, 64)
return int(val), err
}
func (p *Parser) parseArray() ([]any, error) {
p.nextToken() // consume [
arr := make([]any, 0)
for p.curToken.Type != TokenRBracket {
if p.curToken.Type == TokenNewline {
p.nextToken()
continue
}
val, err := p.parseValue()
if err != nil {
return nil, err
}
arr = append(arr, val)
if p.curToken.Type == TokenComma {
p.nextToken()
} else if p.curToken.Type != TokenRBracket {
// Check for newlines between elements if missing comma? TOML usually requires comma.
// Relaxed parser: require comma unless followed immediately by bracket (trailing comma allowed)
if p.curToken.Type == TokenNewline {
p.nextToken()
continue
}
return nil, fmt.Errorf("expected comma or closing bracket in array at line %d", p.curToken.Line)
}
}
p.nextToken() // consume ]
return arr, nil
}
func (p *Parser) parseInlineTable() (map[string]any, error) {
p.nextToken() // consume {
m := make(map[string]any)
for p.curToken.Type != TokenRBrace {
if p.curToken.Type == TokenNewline {
p.nextToken()
continue
}
// Parse key = value
keys, err := p.parseKeyParts()
if err != nil {
return nil, err
}
if p.curToken.Type != TokenEqual {
return nil, fmt.Errorf("expected '=' in inline table at line %d", p.curToken.Line)
}
p.nextToken()
val, err := p.parseValue()
if err != nil {
return nil, err
}
// Inline tables can have dotted keys too: { a.b = 1 }
if err := p.assignValue(m, keys, val); err != nil {
return nil, err
}
if p.curToken.Type == TokenComma {
p.nextToken()
} else if p.curToken.Type != TokenRBrace {
if p.curToken.Type == TokenNewline {
p.nextToken()
continue
}
return nil, fmt.Errorf("expected comma or closing brace in inline table at line %d", p.curToken.Line)
}
}
p.nextToken() // consume }
// Mark inline table immutable. Value.Pointer for maps is documented
// stable for identity comparison. Nested inline tables self-mark on return;
// same-table dotted assignments happen before the mark, so intra-table
// dotted keys ({a.b = 1}) remain unaffected.
p.frozen[reflect.ValueOf(m).Pointer()] = true
return m, nil
}
+54
View File
@@ -0,0 +1,54 @@
package toml
import (
"fmt"
)
// TokenType represents the type of a lexical token
type TokenType int
const (
TokenError TokenType = iota
TokenEOF
TokenComment
// Literals
TokenIdent // bare key
TokenString // "quoted"
TokenInteger // 123
TokenFloat // 123.45
TokenBool // true/false
// Operators and Delimiters
TokenEqual // =
TokenDot // .
TokenComma // ,
TokenLBracket // [
TokenRBracket // ]
TokenLBrace // {
TokenRBrace // }
TokenNewline // \n
)
// Token represents a lexical token
type Token struct {
Type TokenType
Literal string
Line int
Col int
}
func (t Token) String() string {
switch t.Type {
case TokenEOF:
return "EOF"
case TokenError:
return fmt.Sprintf("Error(%s)", t.Literal)
case TokenNewline:
return "Newline"
}
if len(t.Literal) > 20 {
return fmt.Sprintf("%q...", t.Literal[:20])
}
return fmt.Sprintf("%q", t.Literal)
}
+484
View File
@@ -0,0 +1,484 @@
package toml
import (
"testing"
)
// TestUnmarshal_Complex verifies the full pipeline from TOML string to struct
// utilizing the latest generic decoding logic.
func TestUnmarshal_Complex(t *testing.T) {
input := []byte(`
title = "Vi-Fighter Config"
[settings]
debug = true
max_fps = 144
scale = 1.5
[owner]
name = "Admin"
id = 55
[network]
hosts = ["10.0.0.1", "10.0.0.2"]
ports = [8080, 8081]
[[servers]]
name = "alpha"
active = true
[[servers]]
name = "beta"
active = false
`)
type Settings struct {
Debug bool `toml:"debug"`
MaxFPS int `toml:"max_fps"`
Scale float64 `toml:"scale"`
}
type Server struct {
Name string `toml:"name"`
Active bool `toml:"active"`
}
type Config struct {
Title string `toml:"title"`
Settings Settings `toml:"settings"`
Owner map[string]any `toml:"owner"` // Test dynamic map
Network struct {
Hosts []string `toml:"hosts"`
Ports []int `toml:"ports"`
} `toml:"network"`
Servers []Server `toml:"servers"` // Test Array of Tables
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
// 1. Basic Fields
if cfg.Title != "Vi-Fighter Config" {
t.Errorf("Title mismatch: got %q", cfg.Title)
}
// 2. Nested Struct & Types
if !cfg.Settings.Debug {
t.Error("Settings.Debug should be true")
}
if cfg.Settings.MaxFPS != 144 {
t.Errorf("Settings.MaxFPS mismatch: got %d", cfg.Settings.MaxFPS)
}
if cfg.Settings.Scale != 1.5 {
t.Errorf("Settings.Scale mismatch: got %f", cfg.Settings.Scale)
}
// 3. Dynamic Map (owner)
if name, ok := cfg.Owner["name"].(string); !ok || name != "Admin" {
t.Errorf("Owner.Name mismatch: got %v", cfg.Owner["name"])
}
// Check int conversion in dynamic map (parser returns int/float, decode handles struct fields, but map keeps raw parser types)
// Parser likely returns int for 55.
if id, ok := cfg.Owner["id"].(int); !ok || id != 55 {
// Fallback check if parser returned generic float for number
if fId, okf := cfg.Owner["id"].(float64); !okf || fId != 55 {
t.Errorf("Owner.ID mismatch: got %T %v", cfg.Owner["id"], cfg.Owner["id"])
}
}
// 4. Slices
if len(cfg.Network.Hosts) != 2 || cfg.Network.Hosts[0] != "10.0.0.1" {
t.Errorf("Network.Hosts mismatch: %v", cfg.Network.Hosts)
}
if len(cfg.Network.Ports) != 2 || cfg.Network.Ports[1] != 8081 {
t.Errorf("Network.Ports mismatch: %v", cfg.Network.Ports)
}
// 5. Array of Tables
if len(cfg.Servers) != 2 {
t.Fatalf("Expected 2 servers, got %d", len(cfg.Servers))
}
if cfg.Servers[0].Name != "alpha" || !cfg.Servers[0].Active {
t.Errorf("Server[0] mismatch: %+v", cfg.Servers[0])
}
if cfg.Servers[1].Name != "beta" || cfg.Servers[1].Active {
t.Errorf("Server[1] mismatch: %+v", cfg.Servers[1])
}
}
// TestDecode_RawPrimitives validates the reflection logic in decode.go
// specifically for type coercion (int -> float, int -> int64, etc.)
func TestDecode_RawPrimitives(t *testing.T) {
// Simulate map[string]any output from Parser
data := map[string]any{
"int_val": 100, // int
"float_val": 123.45, // float64
"bool_val": true, // bool
"str_val": "hello", // string
"any_val": "dynamic", // string -> any
}
type Target struct {
Int int64 `toml:"int_val"` // Test int -> int64
Float float32 `toml:"float_val"` // Test float64 -> float32
Bool bool `toml:"bool_val"`
Str string `toml:"str_val"`
Any any `toml:"any_val"`
}
var tgt Target
if err := Decode(data, &tgt); err != nil {
t.Fatalf("Decode failed: %v", err)
}
if tgt.Int != 100 {
t.Errorf("Int64 coercion failed: got %d", tgt.Int)
}
// Approximate float comparison
if tgt.Float < 123.44 || tgt.Float > 123.46 {
t.Errorf("Float32 coercion failed: got %f", tgt.Float)
}
if !tgt.Bool {
t.Error("Bool failed")
}
if tgt.Str != "hello" {
t.Error("String failed")
}
if tgt.Any != "dynamic" {
t.Error("Any interface assignment failed")
}
}
// TestDecode_NestedStructs tests direct Decode usage without Parser
func TestDecode_NestedStructs(t *testing.T) {
// Nested map structure simulating [parent.child]
data := map[string]any{
"parent": map[string]any{
"child": map[string]any{
"val": 99,
},
},
}
type Child struct {
Val int `toml:"val"`
}
type Parent struct {
Child Child `toml:"child"`
}
type Top struct {
Parent Parent `toml:"parent"`
}
var tgt Top
if err := Decode(data, &tgt); err != nil {
t.Fatalf("Decode nested failed: %v", err)
}
if tgt.Parent.Child.Val != 99 {
t.Errorf("Nested decoding failed: got %d", tgt.Parent.Child.Val)
}
}
// TestDecode_SliceCoercion tests converting []any (from parser) to specific slices
func TestDecode_SliceCoercion(t *testing.T) {
data := map[string]any{
"nums": []any{1, 2, 3},
}
type T struct {
Nums []int `toml:"nums"`
}
var tgt T
if err := Decode(data, &tgt); err != nil {
t.Fatalf("Decode slice failed: %v", err)
}
if len(tgt.Nums) != 3 || tgt.Nums[2] != 3 {
t.Errorf("Slice decoding failed: %v", tgt.Nums)
}
}
// TestDecode_MapMap tests map[string]map[string]T
func TestDecode_MapMap(t *testing.T) {
data := map[string]any{
"config": map[string]any{
"env": map[string]any{
"production": true,
},
},
}
type T struct {
Config map[string]map[string]bool `toml:"config"`
}
var tgt T
if err := Decode(data, &tgt); err != nil {
t.Fatalf("Decode map-map failed: %v", err)
}
if !tgt.Config["env"]["production"] {
t.Error("Deep map decoding failed")
}
}
// TestDecode_TargetValidation ensures non-pointer targets fail
func TestDecode_TargetValidation(t *testing.T) {
var tgt struct{}
err := Decode(map[string]any{}, tgt) // Pass by value (error)
if err == nil {
t.Error("Expected error when passing non-pointer to Decode")
}
var ptr *struct{} = nil
err = Decode(map[string]any{}, ptr) // Pass nil pointer (error)
if err == nil {
t.Error("Expected error when passing nil pointer to Decode")
}
}
// TestDecode_PrivateHelperAccess verifies toFloat functionality indirectly
// via Decode since we are in package toml
func TestDecode_TypeMismatch(t *testing.T) {
data := map[string]any{
"val": "not a number",
}
type T struct {
Val int `toml:"val"`
}
var tgt T
err := Decode(data, &tgt)
if err == nil {
t.Error("Expected error decoding string to int")
}
}
func TestLexer_DottedKeyVsFloat(t *testing.T) {
// Verify lexer correctly distinguishes dotted keys from floats
tests := []struct {
input string
expected []TokenType
}{
{"a.b", []TokenType{TokenIdent, TokenDot, TokenIdent, TokenEOF}},
{"1.5", []TokenType{TokenFloat, TokenEOF}},
{"-3.14", []TokenType{TokenFloat, TokenEOF}},
{"+2.0", []TokenType{TokenFloat, TokenEOF}},
{"a.b.c", []TokenType{TokenIdent, TokenDot, TokenIdent, TokenDot, TokenIdent, TokenEOF}},
{"1e10", []TokenType{TokenFloat, TokenEOF}},
{"1.5e-3", []TokenType{TokenFloat, TokenEOF}},
{"key_name", []TokenType{TokenIdent, TokenEOF}},
{"key-name", []TokenType{TokenIdent, TokenEOF}},
}
for _, tc := range tests {
l := NewLexer([]byte(tc.input))
var got []TokenType
for {
tok := l.NextToken()
got = append(got, tok.Type)
if tok.Type == TokenEOF || tok.Type == TokenError {
break
}
}
if len(got) != len(tc.expected) {
t.Errorf("input %q: token count mismatch, got %d, want %d", tc.input, len(got), len(tc.expected))
continue
}
for i, tt := range tc.expected {
if got[i] != tt {
t.Errorf("input %q: token[%d] = %v, want %v", tc.input, i, got[i], tt)
}
}
}
}
func TestUnmarshal_FloatInNestedTable(t *testing.T) {
input := []byte(`
[physics.gravity]
x = 0.0
y = -9.81
z = 0.0
`)
type Vec3 struct {
X float64 `toml:"x"`
Y float64 `toml:"y"`
Z float64 `toml:"z"`
}
type Config struct {
Physics struct {
Gravity Vec3 `toml:"gravity"`
} `toml:"physics"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.Physics.Gravity.Y != -9.81 {
t.Errorf("Gravity.Y = %f, want -9.81", cfg.Physics.Gravity.Y)
}
}
func TestUnmarshal_DeepDottedKeys(t *testing.T) {
input := []byte(`
[a.b.c.d]
value = 42
`)
type Config struct {
A struct {
B struct {
C struct {
D struct {
Value int `toml:"value"`
} `toml:"d"`
} `toml:"c"`
} `toml:"b"`
} `toml:"a"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.A.B.C.D.Value != 42 {
t.Errorf("Value = %d, want 42", cfg.A.B.C.D.Value)
}
}
func TestUnmarshal_MixedDottedAndInline(t *testing.T) {
input := []byte(`
[server.http]
port = 8080
tls = { enabled = true, cert = "server.crt" }
`)
type TLS struct {
Enabled bool `toml:"enabled"`
Cert string `toml:"cert"`
}
type Config struct {
Server struct {
HTTP struct {
Port int `toml:"port"`
TLS TLS `toml:"tls"`
} `toml:"http"`
} `toml:"server"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.Server.HTTP.Port != 8080 {
t.Errorf("Port = %d, want 8080", cfg.Server.HTTP.Port)
}
if !cfg.Server.HTTP.TLS.Enabled {
t.Error("TLS.Enabled should be true")
}
}
func TestUnmarshal_ScientificNotation(t *testing.T) {
input := []byte(`
planck = 6.626e-34
avogadro = 6.022e+23
speed_of_light = 3e8
`)
type Config struct {
Planck float64 `toml:"planck"`
Avogadro float64 `toml:"avogadro"`
SpeedOfLight float64 `toml:"speed_of_light"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.SpeedOfLight != 3e8 {
t.Errorf("SpeedOfLight = %e, want 3e8", cfg.SpeedOfLight)
}
}
func TestUnmarshal_HyphenatedKeys(t *testing.T) {
input := []byte(`
[my-section]
my-key = "value"
another_key = 123
`)
type Config struct {
MySection struct {
MyKey string `toml:"my-key"`
AnotherKey int `toml:"another_key"`
} `toml:"my-section"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.MySection.MyKey != "value" {
t.Errorf("MyKey = %q, want \"value\"", cfg.MySection.MyKey)
}
}
func TestUnmarshal_ArrayOfTablesWithPointers(t *testing.T) {
input := []byte(`
[[items]]
name = "first"
value = 1.5
[[items]]
name = "second"
value = 2.5
`)
type Item struct {
Name string `toml:"name"`
Value float64 `toml:"value"`
}
type Config struct {
Items []*Item `toml:"items"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if len(cfg.Items) != 2 {
t.Fatalf("len(Items) = %d, want 2", len(cfg.Items))
}
if cfg.Items[0] == nil || cfg.Items[0].Value != 1.5 {
t.Errorf("Items[0] mismatch: %+v", cfg.Items[0])
}
}
func TestUnmarshal_NestedMapPointers(t *testing.T) {
input := []byte(`
[entities.player]
health = 100
speed = 5.5
[entities.enemy]
health = 50
speed = 3.0
`)
type Entity struct {
Health int `toml:"health"`
Speed float64 `toml:"speed"`
}
type Config struct {
Entities map[string]*Entity `toml:"entities"`
}
var cfg Config
if err := Unmarshal(input, &cfg); err != nil {
t.Fatalf("Unmarshal failed: %v", err)
}
if cfg.Entities["player"] == nil || cfg.Entities["player"].Speed != 5.5 {
t.Errorf("player mismatch: %+v", cfg.Entities["player"])
}
if cfg.Entities["enemy"] == nil || cfg.Entities["enemy"].Health != 50 {
t.Errorf("enemy mismatch: %+v", cfg.Entities["enemy"])
}
}