Version: 1.2.3
Author: Jane Doe (jane.doe@example.com)
License: MIT
Repo: GitHub
Table of Contents
- Introduction
- Core Features
- Installation
- Quickstart Guide
- Advanced Usage
- API Reference
- Extending Dirpy: Custom Path Processors
- Security Considerations
- Performance and Benchmarking
- Test Cases
- License and Contribution
- Conclusion
- Frequently Asked Questions (FAQ)
1. Introduction
Dirpy is a lightweight, cross‑platform Python library that provides a high‑level API for working with directories and the file system. It abstracts away the differences between POSIX and Windows environments, allowing developers to write portable code for tasks such as creating, listing, copying, moving, and deleting directories, as well as monitoring changes in real time. Dirpy also offers integration points for remote file systems (SFTP, SCP, HTTP) and a plugin system for extending its functionality.
2. Core Features
Key capabilities of Dirpy include:
- Cross‑platform directory and file management
- Event‑driven monitoring of directory changes
- Remote adapters for SFTP, SCP, and HTTP storage
- Plugin architecture for custom path processing
- Safe path validation and permission preservation
- Extensive unit tests and continuous integration support
3. Installation
Dirpy can be installed directly from PyPI using pip:
pip install dirpy
For the optional inotify support on Linux, include the [extra] feature:
pip install dirpy[extra]
4. Quickstart Guide
Creating a Directory
Dirpy makes it easy to create a directory tree with optional template files.
from dirpy import Directory
root = Directory("/tmp/project")
root.create(template="basic") # creates /tmp/project with default files
Listing Entries
Use list() to enumerate files and subdirectories.
for entry in root.list():
print(entry.path)
Copying and Moving
Copy or move a directory while preserving metadata.
backup = root.copy_to("/tmp/backup", preserve_metadata=True)
Monitoring Changes
Attach event handlers to react to file system events.
def on_change(event):
print(f"Detected {event['type']} at {event['path']}")
root.on(event_type="modified", handler=on_change)
5. Advanced Usage
Remote Adapters
Dirpy includes adapters for SFTP, SCP, and HTTP storage. Example:
from dirpy.remote import SFTPAdapter
sftp = SFTPAdapter(host="example.com", username="user", password="pass")
sftp.upload("/local/file.txt", "/remote/path/file.txt")
Custom Path Processors
Plugins allow developers to inject custom logic into Dirpy’s operations. For example, a processor can enforce security policies or modify paths on the fly.
Directory Class
class Directory(path: str, *, follow_links: bool = False, root: Optional[str] = None)
- path – Absolute path to the directory.
- follow_links – Whether to traverse symbolic links.
- root – Optional root for path validation.
create(template: Optional[str] = None)– Create directory tree; optional template for initial files.list(follow_links: bool = False)– Enumerate directory entries.copyto(dest: str, preservemetadata: bool = True, follow_links: bool = False)– Recursively copy.moveto(dest: str, preservemetadata: bool = True)– Move directory.remove(follow_links: bool = False)– Delete directory tree.on(event_type: str, handler: Callable)– Register event listener.hash_tree(algorithm: str = "sha256")– Compute a hash of the directory tree.
Remote Adapters
Available adapters include SFTPAdapter, SCPAdapter, and HTTPAdapter. Each exposes upload and download methods.
7. Extending Dirpy: Custom Path Processors
Dirpy provides a flexible system for adding custom behavior to its core operations. Below is a deeper dive into the design and usage of path processors, including advanced examples that showcase the library’s extensibility.
Plugin Registration
Plugins are registered on the Directory class through a static method. Once registered, all subsequent directory instances will automatically invoke the plugin during construction. A plugin may also be unregistered, allowing developers to enable or disable functionality dynamically.
class Directory:
_plugins = []
@staticmethod
def register_plugin(plugin):
Directory._plugins.append(plugin)
@staticmethod
def unregister_plugin(plugin):
Directory._plugins.remove(plugin)
Example Processors
Below are two processors you can drop into your project to illustrate common use cases.
- sanitize_path – Rejects any path that contains unsafe substrings.
- expandenvvars – Replaces placeholders like
$HOMEwith the actual environment variable value.
import re
from dirpy import Directory
def sanitize_path(dir_obj):
# Reject paths containing double slashes (e.g., /tmp//project)
if re.search(r'//', dir_obj.path):
raise ValueError(f"Invalid path: {dir_obj.path}")
def expand_env_vars(dir_obj):
# Replace $HOME with the user's home directory
if "$HOME" in dir_obj.path:
dir_obj.path = dir_obj.path.replace("$HOME", os.path.expanduser("~"))
Directory.register_plugin(sanitize_path)
Directory.register_plugin(expand_env_vars)
Using Plugins in Practice
Once plugins are registered, they automatically influence any Directory creation.
# Assume $HOME is set in the environment
proj = Directory("$HOME/myproject")
# sanitize_path validates the path, expand_env_vars resolves it
proj.create()
8. Security Considerations
Dirpy includes several safety features that help protect against common file system vulnerabilities.
- Safe path validation ensures that all operations occur within a specified root directory.
- All remote adapters enforce the use of secure protocols (SFTP, SCP, HTTPS) and require authentication credentials.
- Permissions are preserved by default during copy and move operations, and optionally restricted by the user.
9. Performance and Benchmarking
Dirpy’s performance has been evaluated on both Windows and Linux. Key metrics include:
- Creating a 1‑K level deep directory tree: ~15 ms on average.
- Listing entries for a directory containing 10 000 files: ~45 ms.
- Recursive copy of a 50 MB directory: ~600 ms.
These benchmarks were performed on a modern laptop (Intel i7, 16 GB RAM). Results may vary on slower hardware or over networked file systems.
10. Test Cases
Dirpy ships with a comprehensive test suite located under tests/. Below are a few illustrative examples that can be dropped into your own project or used as a starting point for creating new tests. All tests rely on the pytest framework.
tests/test_directory.py
import os
import tempfile
import pytest
from dirpy import Directory
# Helper to generate temporary directories
@pytest.fixture
def tmp_dir(tmp_path):
dir_path = os.path.join(tmp_path, "demo")
return Directory(dir_path)
def test_create_and_list(tmp_dir):
"""Verify that a directory can be created and its entries enumerated."""
tmp_dir.create(template="empty") # template adds no files
assert os.path.isdir(tmp_dir.path)
entries = list(tmp_dir.list())
# After creating an empty directory, we should see no entries
assert len(entries) == 0
def test_copy_to(tmp_dir):
"""Test that copy_to preserves file contents and metadata."""
# Setup: create a subdirectory with a file
sub = Directory(os.path.join(tmp_dir.path, "sub"))
sub.create()
file_path = os.path.join(sub.path, "test.txt")
with open(file_path, "w") as f:
f.write("Hello, World!")
# Copy
dest = tmp_dir.copy_to(os.path.join(tmp_dir.path, "copy"), preserve_metadata=True)
copied_file = os.path.join(dest.path, "sub", "test.txt")
assert os.path.isfile(copied_file)
with open(copied_file) as f:
assert f.read() == "Hello, World!"
def test_event_listener(tmp_dir):
"""Ensure that event handlers are called on modification."""
events = []
def handler(event):
events.append(event)
tmp_dir.on(event_type="created", handler=handler)
# Trigger a file creation
test_file = os.path.join(tmp_dir.path, "new.txt")
with open(test_file, "w") as f:
f.write("test")
# Small delay to allow the event loop to process
import time
time.sleep(0.1)
assert any(ev["path"] == test_file for ev in events)
def test_remote_sftp_stub(tmp_dir):
"""Dummy test that demonstrates usage of the SFTP adapter."""
from dirpy.remote import SFTPAdapter
# Note: This test does not actually connect to a server.
# It's intended as an example of how you would call the adapter.
sftp = SFTPAdapter(host="localhost", username="user", password="pass")
# We simply ensure that the object can be instantiated.
assert isinstance(sftp, SFTPAdapter)
# To actually test upload/download you would mock paramiko or
# use a real SFTP server in integration tests.
Running the test suite:
pytest --maxfail=1 --disable-warnings
8. License and Contribution
Dirpy is released under the MIT license. Contributions are welcome! Please follow the standard GitHub workflow: fork the repository, create a feature branch, submit a pull request, and run the CI tests before merging.
9. Conclusion
Dirpy strikes a balance between simplicity and extensibility, making it an ideal choice for developers who need reliable directory management across multiple operating systems. Its event‑driven architecture, remote adapters, and plugin system give it the flexibility to adapt to a wide variety of use cases, from building project scaffolds to synchronizing data with cloud storage.
10. Frequently Asked Questions (FAQ)
Here are some of the most common questions developers have about Dirpy.
What operating systems are supported?
Dirpy works on Windows, macOS, and Linux. The core functionality is platform‑agnostic; inotify support on Linux is optional.
How do I handle symbolic links?
Use the follow_links parameter when creating a Directory instance or calling methods like list and copy_to.
Can I use Dirpy with remote file systems?
Yes. Dirpy ships with adapters for SFTP, SCP, and HTTP. Refer to the Remote Adapters section for usage examples.
Is there a way to plug in custom logic into Dirpy’s operations?
Dirpy includes a plugin system via Directory.register_plugin. See the Extending Dirpy section for details.
How does Dirpy ensure security when performing file operations?
Dirpy performs safe path validation against an optional root and preserves permissions by default during copy and move operations.
What are the recommended best practices for using Dirpy in production?
- Always set a
rootparameter to prevent accidental deletion of critical system directories. - Enable
preserve_metadataunless you have a specific reason to override it. - For Linux, use the
[extra]feature to get fast inotify support. - Use the plugin system to enforce custom security policies if needed.
11. Test Cases
Below are a few representative test cases that illustrate how Dirpy’s API can be exercised and verified. These tests are written with pytest and can be found in the tests/ directory of the repository.
tests/test_directory_creation.py
import os
import tempfile
import pytest
from dirpy import Directory
def test_basic_create():
with tempfile.TemporaryDirectory() as tmp:
dir_path = os.path.join(tmp, "project")
dir_obj = Directory(dir_path)
dir_obj.create()
assert os.path.isdir(dir_obj.path)
def test_template_create():
with tempfile.TemporaryDirectory() as tmp:
dir_path = os.path.join(tmp, "project")
dir_obj = Directory(dir_path)
dir_obj.create(template="basic") # create with a README
assert os.path.isfile(os.path.join(dir_obj.path, "README.md"))
tests/test_directory_listing.py
import os
import tempfile
import pytest
from dirpy import Directory
def test_listing_empty(tmp_path):
dir_obj = Directory(os.path.join(tmp_path, "empty"))
dir_obj.create()
assert list(dir_obj.list()) == []
def test_listing_with_files(tmp_path):
dir_obj = Directory(os.path.join(tmp_path, "folder"))
dir_obj.create()
# Add files
filenames = ["a.txt", "b.txt", "c.txt"]
for name in filenames:
open(os.path.join(dir_obj.path, name), "w").close()
entries = [e.path for e in dir_obj.list()]
assert set(entries) == set(os.path.join(dir_obj.path, f) for f in filenames)
tests/test_directory_copy.py
import os
import tempfile
import pytest
from dirpy import Directory
def test_copy_preserves_contents(tmp_path):
src_dir = os.path.join(tmp_path, "src")
dst_dir = os.path.join(tmp_path, "dst")
# Setup source
dir_obj = Directory(src_dir)
dir_obj.create()
file_path = os.path.join(dir_obj.path, "file.txt")
with open(file_path, "w") as f:
f.write("sample")
# Copy
copied = dir_obj.copy_to(dst_dir)
copied_file = os.path.join(copied.path, "file.txt")
assert os.path.isfile(copied_file)
with open(copied_file) as f:
assert f.read() == "sample"
tests/test_directory_events.py
import time
import pytest
from dirpy import Directory
def test_event_on_file_creation(tmp_path):
dir_obj = Directory(os.path.join(tmp_path, "event"))
events = []
def listener(event):
events.append(event)
dir_obj.on(event_type="created", handler=listener)
dir_obj.create()
file_path = os.path.join(dir_obj.path, "new.txt")
with open(file_path, "w") as f:
f.write("data")
time.sleep(0.05) # wait for event loop
assert any(ev["path"] == file_path for ev in events)
Running the full suite with coverage:
pytest --cov=dirpy
These tests cover typical use cases such as creation, listing, copying, and event handling. They also demonstrate how to structure tests for new features or bug fixes.
Happy coding!
No comments yet. Be the first to comment!