Add2dir

Introduction

add2dir is a command‑line utility designed to simplify the process of adding files to a target directory within a project’s source tree. The tool operates by copying or moving specified source files into a destination directory, optionally performing file name transformations, filtering, and metadata preservation. add2dir is commonly used in build pipelines, deployment scripts, and continuous integration workflows where the organization of artifacts is critical for reproducibility and consistency. The utility is distributed as a standalone executable that can be invoked on a variety of operating systems, including Unix‑like systems and Windows, through a consistent interface.

Although add2dir is not a component of the GNU Core Utilities, it shares similar philosophical principles: simplicity, minimal dependencies, and a focus on achieving a single, well‑defined task efficiently. The tool is available under an open‑source license, encouraging community contributions and adaptation to specialized contexts such as embedded systems, scientific computing, and web application packaging.

History and Background

The concept of a file‑copying utility tailored for directory organization emerged in the late 2000s, when developers began to experience increasing complexity in managing build artifacts across multi‑module projects. Existing tools such as cp and rsync, while powerful, required extensive scripting to accommodate the specific requirements of build systems, especially when dealing with pattern matching, metadata preservation, and cross‑platform compatibility.

add2dir was first released in 2011 by a small team of developers working on a large-scale web application. The initial release provided basic functionality: copying a set of files into a specified directory while preserving timestamps and ownership information. Over subsequent releases, the utility expanded to include options for filtering files by extension, renaming patterns, and integrating with popular build tools such as Make, CMake, and Gradle.

Throughout its development, the add2dir project has maintained a lean codebase written in C++, with minimal external dependencies. This design choice was motivated by the desire to keep the utility portable and efficient, enabling its deployment in environments with constrained resources or strict security requirements.

Architecture and Design

Core Functionality

The core of add2dir is a lightweight library that abstracts file system operations. It exposes an interface for specifying source paths, destination directories, and transformation rules. Internally, the library performs the following steps in order:

Parse command‑line arguments and validate input.
Resolve relative and absolute paths, applying user‑specified working directories.
Determine the set of source files by expanding glob patterns.
Apply optional filters and renaming rules.
Perform the copy or move operation with optional metadata preservation.
Report status and errors through a standardized output format.

Modular Design

add2dir adopts a modular architecture that separates concerns into distinct components: ArgumentParser, PathResolver, FileFilter, FileRenamer, FileCopier, and Logger. Each module is responsible for a single aspect of the overall process, facilitating unit testing, maintenance, and the potential for future extensions such as remote file handling or integration with cloud storage services.

Cross‑Platform Considerations

The implementation relies on the C++17 standard library for file system operations, ensuring consistent behavior across operating systems. The library abstracts platform‑specific details such as path separators, case sensitivity, and permission models. For example, on Windows, add2dir translates POSIX‑style paths to the native format and uses the Win32 API to preserve timestamps and attributes. On Unix‑like systems, the utility leverages POSIX calls to maintain ownership and permission bits.

Syntax and Options

Basic Invocation

The simplest form of the add2dir command copies a single file to a destination directory:

add2dir source_file destination_directory

In this form, the source file is copied into the destination directory, retaining its original name.

Multiple Sources and Globbing

add2dir accepts multiple source arguments, and each can include glob patterns. For instance:

add2dir src/**/*.js build/js

All JavaScript files under the src directory will be copied to the build/js directory, preserving the relative subdirectory structure.

Supported Options

The utility offers a range of options that modify its behavior. The following table summarizes the most common options:

Option	Description
-m, --move	Move files instead of copying.
-f, --filter <pattern>	Filter source files by a shell‑style pattern.
-r, --rename <template>	Rename files using a template string with placeholders.
-p, --preserve	Preserve timestamps, ownership, and permissions.
-v, --verbose	Print detailed progress information.
-q, --quiet	Suppress non‑essential output.
-h, --help	Display help information.

Renaming Templates

The rename option uses a simple templating syntax. Placeholders are enclosed in curly braces. The following example prepends a timestamp to each file name:

add2dir -r '{date:%Y%m%d}_{name}{ext}' -p src/*.txt backup

Here, {date:%Y%m%d} is replaced by the current date in the specified format, {name} is the base file name, and {ext} is the file extension.

Common Use Cases

Build Artifact Organization

In large software projects, build systems generate a multitude of intermediate files. add2dir helps maintain a clean output directory by selectively moving compiled objects, documentation, and resource files into their designated locations. For example, a CMake project may use add2dir to consolidate all header files into a single include directory after generation.

Deployment Packaging

When preparing a deployment package, add2dir can gather configuration files, binaries, and static assets into a structure expected by the target environment. The ability to rename and filter files allows developers to strip sensitive information or conform to naming conventions required by third‑party services.

Data Migration

Data migration scripts often need to reorganize files from legacy systems into modern schemas. add2dir provides a declarative way to move data files while applying transformations to file names, ensuring that downstream applications can locate the data without manual intervention.

Continuous Integration (CI) Pipelines

CI tools such as Jenkins, GitHub Actions, and GitLab CI frequently employ add2dir in build scripts to ensure that artifact directories remain consistent across runs. The command’s non‑interactive nature and robust error handling make it suitable for automated pipelines that require deterministic outcomes.

Integration with Build Systems

Make

In Makefiles, add2dir can be invoked as a recipe step. A typical rule might look like:

.PHONY: prepare
prepare:
    add2dir src/*.h include/
    add2dir -m bin/* build/

This rule copies header files into an include directory and moves binaries into a build directory.

CMake

CMake provides a custom command interface that can execute add2dir during the build process. For example:

add_custom_command(
    OUTPUT ${CMAKE_BINARY_DIR}/generated/include
    COMMAND add2dir -f '*.h' -p ${CMAKE_CURRENT_SOURCE_DIR}/src ${CMAKE_BINARY_DIR}/generated/include
    DEPENDS ${SOURCE_FILES}
    COMMENT "Collect header files into generated include directory")

Here, the custom command is dependent on source files, ensuring that changes trigger regeneration.

Gradle

In Gradle, the exec task can run add2dir as part of a build script:

task collectHeaders(type: Exec) {
    commandLine 'add2dir', '-f', '**/*.java', '-p', "${projectDir}/src/main/java", "${buildDir}/generated-sources"
}

Gradle’s up‑to‑date checks can avoid re‑execution when input files are unchanged.

Variants and Implementations

Python Wrapper

A Python wrapper around the add2dir binary allows developers to invoke the utility from Python scripts, capturing output and integrating with Python’s rich ecosystem. The wrapper uses the subprocess module to launch the binary and parses its output into structured data for further processing.

Rust Binding

For projects written in Rust, a crate named add2dir-rs provides a native Rust API that mirrors the command‑line interface. The crate exposes safe abstractions for file operations, filtering, and renaming, making it convenient to embed add2dir functionality into larger Rust applications.

Node.js Module

The add2dir-node module offers a JavaScript API for Node.js applications. It internally spawns the add2dir executable or uses a pure JavaScript implementation when available. The module is used in build tools like Gulp and Webpack to streamline asset management.

Performance and Optimization

Algorithmic Efficiency

add2dir’s core algorithm iterates over the list of source files once, applying filters and renaming rules on the fly. The implementation uses buffered I/O streams to copy file contents, minimizing memory usage even for large files. The tool also supports multi‑threaded copying on supported platforms, distributing work across CPU cores to reduce overall transfer time.

Caching Mechanisms

To avoid redundant operations, add2dir can maintain a lightweight cache file that records metadata of previously processed files. When run again, the utility compares the cache against current file timestamps and skips copies for unchanged files, providing an optimization similar to incremental builds.

Resource Footprint

The binary is typically less than 1 MB in size, with a minimal runtime dependency on the C++17 standard library. Memory usage during execution is bounded by the size of the largest file being processed, as the tool streams data directly from source to destination without loading entire files into RAM.

Security Considerations

Input Validation

add2dir performs strict validation of command‑line arguments. It rejects paths that contain null bytes, control characters, or overly long strings. The utility also checks for relative paths that could potentially escape the intended destination directory, mitigating directory traversal attacks.

Permission Handling

When the preserve option is enabled, add2dir attempts to replicate the ownership and permission bits of source files. On systems where the executing user lacks sufficient privileges, the tool falls back to default permissions while logging a warning. This approach ensures that the operation does not silently fail due to insufficient rights.

Audit and Logging

By default, add2dir logs all operations to standard output. The verbose option provides granular details such as file paths, sizes, and timestamps, aiding in forensic analysis if required. Logging is designed to be tamper‑evident; log entries include timestamps in ISO 8601 format.

Community and Ecosystem

Contributors

The add2dir project has attracted contributors from both industry and academia. Key maintainers include individuals who specialize in build automation, file system programming, and open‑source licensing. Contributions are managed through a standard pull request workflow on a public repository.

Documentation

Comprehensive documentation is available in the form of a user guide, API reference for bindings, and a developer handbook. The documentation emphasizes best practices for integrating add2dir into complex build pipelines and includes troubleshooting sections for common error scenarios.

User Community

Discussion forums and mailing lists host active conversations about feature requests, bug reports, and deployment strategies. Regular community surveys inform the roadmap, ensuring that the tool evolves in response to real‑world needs.

Future Directions

Remote File Support

Planned enhancements include support for copying files over network protocols such as SSH and FTP. The design will incorporate a plugin architecture to abstract transport mechanisms, allowing the same command syntax to operate on both local and remote file systems.

Graphical User Interface

A lightweight GUI application is under consideration to make add2dir accessible to users who prefer a visual workflow. The interface would expose the same options as the command line while providing drag‑and‑drop functionality and progress visualization.

Extended Pattern Matching

Future releases aim to incorporate extended pattern matching features such as regular expressions and semantic versioning filters. This would enhance the tool’s flexibility in environments where complex file selection criteria are required.

Integration with Container Orchestration

As containerization becomes ubiquitous, add2dir will explore integration points with orchestration platforms like Kubernetes and Docker Swarm. This may involve generating volume mounts or preparing container image layers with optimized file organization.

Search

Table of Contents