Introduction
go2convert is a command‑line utility and accompanying Go library designed to facilitate the transformation of data between multiple textual and binary formats. It supports conversions among JSON, XML, YAML, TOML, CSV, and Protocol Buffers, as well as unit conversions for common measurement categories such as temperature, distance, mass, and currency. The tool was conceived to address the recurring need for data migration and normalization in software development, data science, and system integration contexts. It is distributed under a permissive open‑source license and is actively maintained by a community of contributors worldwide.
History and Background
Origins
The project began in 2018 when a group of developers at a data‑integration consultancy identified a lack of flexible, language‑agnostic conversion utilities. Existing solutions were either proprietary, limited in scope, or required complex configuration. The team selected Go (Golang) as the implementation language due to its strong support for static typing, concurrency, and a growing ecosystem of libraries for parsing and emitting various data formats.
Early Releases
The first public release, version 0.1.0, appeared on a popular code‑hosting platform in January 2019. It provided basic JSON–XML conversion, unit conversion for temperature, and a rudimentary command‑line interface. Feedback from early adopters highlighted the need for a comprehensive format catalog and improved error handling.
Maturation
Between 2019 and 2021, the project evolved through a series of iterative releases. New modules were added for CSV, YAML, TOML, and Protocol Buffers. The library gained support for nested data structures, custom field mapping, and schema validation. A plugin system was introduced in version 1.0.0 to allow third‑party developers to extend conversion capabilities without modifying the core codebase.
Current State
As of March 2026, go2convert has achieved a stable major version (2.x) and a large user base across multiple industries. It is used in continuous‑integration pipelines, data‑engineering workflows, and as a learning tool in educational settings. The project's governance model emphasizes community contributions, transparent issue tracking, and regular release cadences.
Key Concepts
Data Formats
go2convert treats each supported format as a first‑class citizen. Supported formats include:
- JSON (JavaScript Object Notation)
- XML (Extensible Markup Language)
- YAML (YAML Ain’t Markup Language)
- TOML (Tom's Obvious, Minimal Language)
- CSV (Comma‑Separated Values)
- Protocol Buffers (binary serialization)
The library provides parsing, validation, and serialization functions for each format. Conversions between formats are performed by deserializing input into an intermediate representation (often a generic map or struct) and re‑serializing it into the target format.
Unit Conversion
go2convert includes a dedicated module for converting values among related units. Supported categories encompass:
- Temperature: Celsius, Fahrenheit, Kelvin
- Distance: meters, kilometers, miles, feet, inches
- Mass: kilograms, grams, pounds, ounces
- Volume: liters, milliliters, gallons, pints
- Currency: conversion based on user‑supplied exchange rates
Conversion functions support arbitrary precision via Go's math/big package, enabling accurate handling of large numbers or high‑resolution requirements.
Configuration and Mapping
Complex conversions often require field renaming, type coercion, or conditional inclusion. go2convert offers a mapping language defined in YAML or JSON that specifies:
- Source and target field names
- Data type transformations (e.g., string to integer)
- Filter expressions based on field values
- Nested mapping for embedded objects or arrays
The mapping language is optional; simple conversions can be performed without it. When a mapping file is supplied, the library applies the defined transformations during the conversion process.
Plugin Architecture
To maintain a lightweight core while allowing extensibility, go2convert implements a plugin system. Plugins are compiled as Go shared objects (.so files) and discovered at runtime. A plugin may provide:
- Support for additional data formats
- Custom conversion logic (e.g., domain‑specific transformations)
- Extended mapping features
The plugin API is documented in the project's reference materials, and several community plugins exist for specialized use cases such as JSON‑to‑Excel conversion and SQL schema generation.
Architecture
Core Library
The core library is organized into several packages, each responsible for a distinct aspect of the tool's functionality:
parser– Implements parsers for each supported format.serializer– Implements serializers for each format.converter– Provides generic conversion functions and manages the mapping language.units– Contains unit conversion logic and currency rate handling.cli– Wraps the core library into a command‑line interface using Go'sflagpackage.
Each package exposes a clean API. For example, the parser package provides a Parse function that accepts a byte slice and returns an interface representing the parsed data. The serializer package provides a corresponding Serialize function.
Intermediate Representation
go2convert uses a generic map[string]interface{} structure as its intermediate representation. This choice enables flexible handling of arbitrary data schemas and simplifies mapping operations. For cases where performance is critical, developers can employ strongly typed structs, but the library will automatically convert between typed and generic representations.
Error Handling
All operations return error values following Go conventions. The library categorizes errors into three primary types:
- Parsing errors – occur when input data does not conform to the expected format.
- Conversion errors – arise during type coercion or mapping violations.
- Runtime errors – include I/O failures, missing plugins, or unsupported formats.
Errors are wrapped with contextual information to aid debugging. The CLI prints errors to standard error and exits with a non‑zero status code.
Concurrency
While most conversion tasks are I/O bound and executed serially, the library exposes a parallel conversion API that accepts a slice of inputs and returns a slice of outputs. Internally, the library uses goroutines and channels to process conversions concurrently, improving throughput for large batch operations.
Usage
Command‑Line Interface
The CLI accepts a set of flags that determine source and target formats, input and output files, mapping files, and other options. A typical invocation looks like this:
go2convert -in data.json -out data.xml -format json xml
Key flags include:
-in– Path to the input file.-out– Path to the output file.-format– One or two format identifiers; the second is optional for conversions.-map– Path to a mapping file.-plugin– Path to a plugin shared object.-log– Path to a log file; defaults to standard error.
When the CLI is invoked without a format specification, it infers the format from file extensions. The CLI also supports reading from standard input and writing to standard output, making it suitable for pipelines.
Library Integration
Developers can import go2convert as a Go module. The library provides a straightforward API for converting data programmatically:
import (
"github.com/example/go2convert"
"io/ioutil"
)
func main() {
input, _ := ioutil.ReadFile("data.yaml")
data, err := go2convert.Parse("yaml", input)
if err != nil { /* handle error */ }
output, err := go2convert.Serialize("json", data)
if err != nil { /* handle error */ }
ioutil.WriteFile("data.json", output, 0644)
}
The library also offers advanced functions such as ConvertWithMapping, which accepts a mapping definition and applies it during conversion.
Unit Conversion Examples
Unit conversion can be performed from the CLI or within code. For example, to convert 100 degrees Celsius to Fahrenheit:
go2convert -unit temp -from celsius -to fahrenheit -value 100
In code:
import "github.com/example/go2convert/units"
func main() {
result, err := units.ConvertTemperature(100, "celsius", "fahrenheit")
if err != nil { /* handle error */ }
fmt.Printf("100°C = %f°F\n", result)
}
Extensions
Plugins
The plugin system allows developers to write extensions in Go and compile them as shared objects. A minimal plugin implements the Plugin interface, which includes methods for registering format handlers, mapping processors, or custom commands. Example plugins include:
- ExcelWriter – exports JSON or CSV data to XLSX format.
- SQLGenerator – produces SQL DDL statements from JSON schemas.
- GraphvizExporter – visualizes data structures as DOT files.
Custom Mappings
Advanced users can write mapping scripts in the project's mapping language, specifying complex transformations. The syntax supports conditional statements, loops, and function calls. A mapping file might look like:
source:
person:
name: firstName
surname: lastName
age: years
target:
individual:
fullName: <<concat("${firstName}", " ", "${lastName}")>>
age: <<${years}>>
Such mappings enable transformations that are not possible with default, flat field copying.
Community and Ecosystem
Contributors
The project hosts a diverse set of contributors, ranging from individual hobbyists to employees of large software firms. Contributions are reviewed through a standard pull‑request workflow. The maintainers emphasize code quality, documentation, and backward compatibility.
Issue Tracking
Issues are managed through an issue tracker that categorizes tickets as bugs, feature requests, documentation, or enhancement. Users are encouraged to provide reproducible examples and descriptive titles to facilitate triage.
Documentation
go2convert offers comprehensive documentation covering:
- Installation instructions for various platforms.
- Command‑line reference with flag explanations.
- API reference for library integration.
- Guidelines for writing plugins and mappings.
- Examples of typical conversion workflows.
The documentation is hosted in Markdown and is automatically rendered into HTML during the release process.
Events
Since 2020, the project has participated in multiple hackathons, conferences, and workshops. Notable appearances include:
- Open Source Summit – live demonstration of the CLI's streaming capabilities.
- Data Engineering Conference – case study on using go2convert in a data‑pipeline.
- GoCon – tutorial on plugin development.
Versions
Release History
go2convert follows semantic versioning. The major release 2.x introduced several architectural improvements:
- Enhanced performance for large files.
- Built‑in support for Protocol Buffers v3.
- Refactored mapping language with stricter validation.
- New plugin API v2 with improved error reporting.
Minor releases typically add new formats, bug fixes, or documentation updates. Patch releases focus on critical security fixes and regression testing.
Change Log
Change logs are published with each release and include:
- New features and enhancements.
- Fixed bugs and their identifiers.
- Known issues and workarounds.
- Deprecation notices for removed APIs.
Security
Vulnerability Management
The project follows a responsible disclosure policy. Security researchers are encouraged to submit findings through a dedicated channel. Vulnerabilities are assigned CVE identifiers when applicable and addressed in subsequent releases.
Sanitization and Validation
During parsing, the library performs strict validation of input data against the expected schema. For formats like XML, the parser disables external entity resolution to prevent XXE attacks. The CLI enforces strict mode by default, rejecting inputs that fail validation.
Dependencies
go2convert relies on well‑maintained third‑party libraries for format support, such as:
- encoding/json – Go's standard JSON library.
- github.com/yalp/jsonpath – for JSON path expressions in mappings.
- github.com/kniren/gota/dataframe – for CSV handling in optional plugins.
All dependencies are audited using the Go modules security scanner.
Contributing
Getting Started
Contributors are guided through the following steps:
- Fork the repository and clone it locally.
- Create a feature branch named
feature/<description>. - Implement changes, add tests, and run the test suite.
- Push changes and open a pull request against the
mainbranch.
The repository includes a CONTRIBUTING.md file that outlines coding standards, commit message conventions, and testing expectations.
Testing
go2convert employs both unit and integration tests. The test suite covers:
- Format parsing and serialization correctness.
- Mapping engine accuracy.
- Plugin loading and execution.
- Unit conversion formulas.
- Command‑line flag parsing.
Continuous integration is configured to run tests on multiple Go versions and operating systems.
License
The project is released under the MIT License, a permissive open‑source license that permits modification, distribution, and commercial use with minimal restrictions. The license text is available in the repository's root directory.
No comments yet. Be the first to comment!