Compare commits

...
12 Commits
7 changed files with 714 additions and 104 deletions
+97 -17
View File
@@ -1,15 +1,20 @@
# dirmd
Generate markdown documentation from directory structure.
Generate markdown documentation from directory structure. Optimized for LLM ingestion.
## Features
- Recursive directory traversal
- Tree visualization (standard tree command format)
- Full file contents with syntax-highlighted code blocks
- ASCII tree visualization (LLM-friendly, minimal token overhead)
- Full file contents with explicit delimiters and metadata
- Filters dotfiles, binaries, and blacklisted extensions
- Vertical slices mode: root file + one markdown per top-level directory
- Append mode for single files or directories
- Force overwrite of existing output files
- Configurable depth, file size limits, ignore patterns
- Optional upload to Proton Drive after generating output
![dirmd](./docs/dirmd.gif)
## Build
@@ -20,23 +25,82 @@ Generate markdown documentation from directory structure.
# Create new documentation
dirmd -o output.md ~/repos/myproject
# Overwrite existing output file
dirmd -o output.md ~/repos/myproject -f
# Append a single file
dirmd -a -i README.md -o existing.md
# Append another directory
dirmd -a ~/repos/anotherproject -o existing.md
# Generate with vertical slices
dirmd -o output.md ~/repos/myproject --vertical-slices
# Generate and upload to Proton Drive (default: /my-files/md/<filename>)
dirmd -o output.md ~/repos/myproject --proton-drive
# Generate and upload to a specific remote path
dirmd -o output.md ~/repos/myproject --proton-drive --drive-path /my-files/docs/api.md
## Vertical Slices Mode
When `--vertical-slices` is passed, dirmd splits output into multiple files instead of one monolithic document:
- **Root file** (`-o output.md`): Full directory tree with cross-references to slice files, a vertical slices index table (file counts, total bytes per directory), and full contents of all root-level files.
- **Slice files** (`cmd.md`, `internal.md`, etc.): One per top-level directory, placed alongside the root file. Each contains its own tree, a back-reference to the root document, and full recursive file contents for that directory.
This mode cannot be combined with `--append` or `--input-file`.
Example output with `dirmd -o docs.md ~/repos/myproject --vertical-slices`:
docs/
├── docs.md ← root: tree + slices index + root file contents
├── cmd.md ← slice: tree + contents of cmd/
└── internal.md ← slice: tree + contents of internal/
## LLM-Optimized Output
Output is designed for RAG ingestion and LLM context windows, not human presentation:
- **ASCII tree format** (`+-` instead of Unicode box-drawing) to reduce token overhead
- **Explicit file delimiters**: `--- FILE: ./path (bytes, lines) ---` and `--- END FILE ---`
- **Document type markers**: `[ROOT]` and `[SLICE]` in H1 headers
- **Cross-references**: Directory entries in root tree show `-> see cmd.md`; slice files reference their root document
- **Slices index table**: File counts and total bytes per slice for informed retrieval decisions
- **Tilde fences** (`~~~`) to avoid conflicts with backticks in source files
## Flags
| Flag | Description | Default |
|------|-------------|---------|
| `-o`, `--output` | Output markdown file path (required) | |
| `-a`, `--append` | Append to existing output file | false |
| `-f`, `--force` | Overwrite existing output file (non-append mode) | false |
| `-i`, `--input-file` | Single file input (append mode only) | |
| `--vertical-slices` | Split output into root file + one markdown per top-level directory | false |
| `--max-size` | Max file size in bytes | 524288 |
| `--max-depth` | Max directory recursion depth | 20 |
| `--ignore` | Additional ignore patterns (glob) | |
| `--extensions` | Additional file extensions to skip | |
| `--proton-drive` | Upload output to Proton Drive after writing locally | false |
| `--drive-path` | Full remote path on Proton Drive (e.g. /my-files/md/report.md) | /my-files/md/&lt;filename&gt; |
## Proton Drive Integration
dirmd can optionally upload the generated markdown file(s) to Proton Drive using the [Proton Drive CLI](https://proton.me/support/drive-cli). The upload is performed only when `--proton-drive` is passed. Without it, dirmd behaves exactly as before — no Proton Drive dependency is required.
In vertical slices mode, all generated files (root + slices) are uploaded to `/my-files/md/`. The `--drive-path` flag is ignored in slices mode; each file is uploaded by its basename.
### Prerequisites
1. Install the Proton Drive CLI — see the [official guide](https://proton.me/support/drive-cli) for download and installation instructions.
2. Authenticate by running `proton-drive auth login`.
3. Ensure `proton-drive` is in your `PATH`.
If the CLI is not installed or you are not logged in, dirmd will report the error to stderr and exit with a non-zero code. Local files are always written successfully before the upload is attempted.
For full documentation on the Proton Drive CLI, including installation, authentication, and troubleshooting, refer to the [official Proton Drive CLI support page](https://proton.me/support/drive-cli).
## Examples
@@ -49,35 +113,51 @@ Generate markdown documentation from directory structure.
# Skip additional file extensions
dirmd -o output.md ~/repos/project --extensions .log --extensions .tmp
# Vertical slices with overwrite
dirmd -o docs.md ~/repos/project -f --vertical-slices
# Generate, then upload to default remote location
dirmd -o docs.md ~/repos/project --proton-drive
# Generate, then upload to a custom remote location
dirmd -o docs.md ~/repos/project --proton-drive --drive-path /my-files/projects/docs.md
# Overwrite existing local file and upload to Drive
dirmd -o docs.md ~/repos/project -f --proton-drive
## Output Format
Produces markdown with:
1. Absolute path as H1 header
2. Directory tree (indented with 4 spaces per level)
3. All text files with content in code blocks
4. Language detection from file extensions
1. Absolute path as H1 header with `[ROOT]` or `[SLICE]` marker
2. ASCII directory tree (indented with `+-` and `|` characters)
3. Vertical slices index table (sliced mode only)
4. All text files with content in fenced code blocks
5. Explicit `--- FILE: ---` and `--- END FILE ---` delimiters with byte and line counts
6. Language detection from file extensions
Example output structure:
Example output structure (non-sliced mode):
# /home/user/repos/myproject
# /home/user/repos/myproject [ROOT]
## Directory Tree
~~~
myproject/
├── cmd/
└── main.go
└── internal/
└── config.go
+- cmd/
| +- main.go
+- internal/
| +- config.go
~~~
## Contents
### ./cmd/main.go
```go
--- FILE: ./cmd/main.go (45 bytes, 3 lines) ---
~~~go
package main
func main() {}
```
~~~
--- END FILE ---
Note: Uses tilde fences (~~~) internally to avoid conflicts with nested backticks in source files.
+103 -24
View File
@@ -17,32 +17,41 @@ var (
rootCmd = &cobra.Command{
Use: "dirmd [flags] <directory>",
Short: "Generate markdown documentation from directory structure",
Long: `dirmd walks a directory and generates a markdown file with:
Long: `dirmd walks a directory and generates markdown files with:
- Directory tree structure
- All readable text files with their contents
Output is always written to a directory specified by -o.
Examples:
# Create new documentation
dirmd -o output.md ~/repos/myproject
# Create new documentation (writes index.md inside ./docs/)
dirmd -o ./docs ~/repos/myproject
# Overwrite existing output directory
dirmd -o ./docs ~/repos/myproject -f
# Append a single file
dirmd -a -i README.md -o existing.md
dirmd -a -i README.md -o ./docs
# Append another directory
dirmd -a -o existing.md ~/repos/anotherproject
# Vertical slices (root index.md + one markdown per top-level dir)
dirmd -o ./docs ~/repos/myproject --vertical-slices
# Generate and upload to Proton Drive (default remote path /my-files/md/<filename>)
dirmd -o output.md ~/repos/myproject --proton-drive
# Skip frontend files
dirmd -o ./docs ~/repos/myproject --skip-frontend
# Generate and upload to Proton Drive
dirmd -o ./docs ~/repos/myproject --proton-drive
# Generate and upload to a specific remote path
dirmd -o output.md ~/repos/myproject --proton-drive --drive-path /my-files/docs/api.md`,
dirmd -o ./docs ~/repos/myproject --proton-drive --drive-path /my-files/docs/api`,
RunE: run,
}
outputPath string
appendMode bool
force bool
singleFile string
maxSize int64
maxDepth int
@@ -50,6 +59,8 @@ Examples:
extensions []string
protonDrive bool
drivePath string
verticalSlices bool
skipFrontend bool
)
func Execute() {
@@ -59,15 +70,18 @@ func Execute() {
}
func init() {
rootCmd.PersistentFlags().StringVarP(&outputPath, "output", "o", "", "output markdown file path (required)")
rootCmd.Flags().BoolVarP(&appendMode, "append", "a", false, "append to existing output file")
rootCmd.PersistentFlags().StringVarP(&outputPath, "output", "o", "", "output directory (required)")
rootCmd.Flags().BoolVarP(&appendMode, "append", "a", false, "append to existing index.md")
rootCmd.Flags().BoolVarP(&force, "force", "f", false, "overwrite existing output directory")
rootCmd.Flags().StringVarP(&singleFile, "input-file", "i", "", "single file input (append mode only)")
rootCmd.Flags().Int64Var(&maxSize, "max-size", config.DefaultMaxSize, "max file size in bytes")
rootCmd.Flags().IntVar(&maxDepth, "max-depth", config.DefaultMaxDepth, "max directory recursion depth")
rootCmd.Flags().StringArrayVar(&ignorePatterns, "ignore", nil, "additional ignore patterns (glob)")
rootCmd.Flags().StringArrayVar(&extensions, "extensions", nil, "additional file extensions to skip (repeatable)")
rootCmd.Flags().BoolVar(&protonDrive, "proton-drive", false, "upload output to Proton Drive after writing locally")
rootCmd.Flags().StringVar(&drivePath, "drive-path", "", "full remote path on Proton Drive (e.g. /my-files/md/report.md). Requires --proton-drive")
rootCmd.Flags().BoolVar(&protonDrive, "proton-drive", false, "upload output directory to Proton Drive after writing locally")
rootCmd.Flags().StringVar(&drivePath, "drive-path", "", "full remote directory on Proton Drive (e.g. /my-files/docs/api). Requires --proton-drive")
rootCmd.Flags().BoolVar(&verticalSlices, "vertical-slices", false, "split output into root index.md + one markdown per top-level directory")
rootCmd.Flags().BoolVar(&skipFrontend, "skip-frontend", false, "omit frontend file types (.html, .css, .js, .ts, .vue, .svelte, etc.)")
rootCmd.MarkPersistentFlagRequired("output")
}
@@ -77,15 +91,31 @@ func run(cmd *cobra.Command, args []string) error {
return fmt.Errorf("--drive-path requires --proton-drive")
}
if verticalSlices && appendMode {
return fmt.Errorf("--vertical-slices cannot be used with --append")
}
if verticalSlices && singleFile != "" {
return fmt.Errorf("--vertical-slices cannot be used with --input-file")
}
excludedExts := append(config.DefaultExts, extensions...)
if skipFrontend {
excludedExts = append(excludedExts, config.FrontendExts...)
}
cfg := &config.Config{
OutputPath: outputPath,
AppendMode: appendMode,
Force: force,
MaxSize: maxSize,
MaxDepth: maxDepth,
Ignores: append(config.DefaultIgnores, ignorePatterns...),
Extensions: append(config.DefaultExts, extensions...),
Extensions: excludedExts,
ProtonDrive: protonDrive,
DrivePath: drivePath,
VerticalSlices: verticalSlices,
SkipFrontend: skipFrontend,
}
if singleFile != "" {
@@ -107,13 +137,26 @@ func run(cmd *cobra.Command, args []string) error {
}
cfg.AbsRoot = abs
// Non-append mode: output must not exist
absOutput, err := filepath.Abs(outputPath)
if err != nil {
return fmt.Errorf("failed to resolve output path: %w", err)
}
absOutput = filepath.Join(filepath.Dir(absOutput), filepath.Base(absOutput)+"_dirmd")
if !appendMode {
if _, err := os.Stat(outputPath); err == nil {
return fmt.Errorf("output file %s already exists (use -a to append)", outputPath)
if _, err := os.Stat(absOutput); err == nil && !force {
return fmt.Errorf("output directory %s already exists (use -f to overwrite)", absOutput)
}
}
if err := os.MkdirAll(absOutput, 0755); err != nil {
return fmt.Errorf("cannot create output directory: %w", err)
}
repoName := filepath.Base(cfg.AbsRoot)
fmt.Printf("%s\n", absOutput)
result, err := walker.Run(cfg)
if err != nil {
return err
@@ -123,16 +166,52 @@ func run(cmd *cobra.Command, args []string) error {
return fmt.Errorf("no readable text files found in %s", cfg.AbsRoot)
}
if err := renderer.Write(result, outputPath, appendMode, cfg.SingleFile); err != nil {
indexBasename := "index_" + repoName + ".md"
if verticalSlices {
generated, err := renderer.WriteSlices(result, absOutput, repoName)
if err != nil {
return err
}
fmt.Fprintf(os.Stderr, "wrote %d entries to %s\n", len(result.Entries), outputPath)
fmt.Fprintf(os.Stderr, "generated %d files in %s\n", len(generated), absOutput)
if protonDrive {
remotePath := drivePath
if remotePath == "" {
remotePath = "/my-files/md/" + filepath.Base(outputPath)
remoteDir := drivePath
if remoteDir == "" {
remoteDir = "/my-files/md/" + filepath.Base(absOutput)
}
if err := drive.CheckAuth(); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
fmt.Fprintln(os.Stderr, "local files were written successfully, but upload was skipped")
return err
}
if err := drive.UploadDir(absOutput, remoteDir); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
fmt.Fprintln(os.Stderr, "local files were written successfully, but upload failed")
return err
}
fmt.Fprintf(os.Stderr, "uploaded %s to %s on Proton Drive\n", absOutput, remoteDir)
}
return nil
}
indexPath := filepath.Join(absOutput, indexBasename)
if err := renderer.Write(result, indexPath, appendMode, cfg.SingleFile); err != nil {
return err
}
fmt.Fprintf(os.Stderr, "wrote %d entries to %s\n", len(result.Entries), indexPath)
if protonDrive {
remoteDir := drivePath
if remoteDir == "" {
remoteDir = "/my-files/md/" + filepath.Base(absOutput)
}
if err := drive.CheckAuth(); err != nil {
@@ -141,13 +220,13 @@ func run(cmd *cobra.Command, args []string) error {
return err
}
if err := drive.Upload(outputPath, remotePath); err != nil {
if err := drive.UploadDir(absOutput, remoteDir); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
fmt.Fprintln(os.Stderr, "local file was written successfully, but upload failed")
return err
}
fmt.Fprintf(os.Stderr, "uploaded %s to %s on Proton Drive\n", outputPath, remotePath)
fmt.Fprintf(os.Stderr, "uploaded %s to %s on Proton Drive\n", absOutput, remoteDir)
}
return nil
@@ -0,0 +1,24 @@
# Automating sending files to proton drive on commits
If you add this to any repos, and have dirmd in path and logged in on proton drive cli, then after each commit the newest code is pushed to proton drive, as vertical slices. If you do not want vertical slices of the code omit --vertical-slices. Also check out the other flags of dirmd if required.
## repos-root/.git/hooks/post-commit
create this file
```sh
#!/bin/bash
set -euo pipefail
if ! command -v dirmd &>/dev/null; then
exit 0
fi
REPO_ROOT=$(git rev-parse --show-toplevel)
REPO_NAME=$(basename "$REPO_ROOT")
dirmd -o "/tmp/${REPO_NAME}_new" "$REPO_ROOT" -f --vertical-slices --proton-drive
```
and then make it executable with something like `chmod +x .git/hooks/post-commit`
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 522 KiB

+39 -5
View File
@@ -24,6 +24,7 @@ var DefaultIgnores = []string{
"dist",
".idea",
".vscode",
"dependency-reduced-pom.xml",
}
var DefaultExts = []string{
@@ -35,12 +36,20 @@ var DefaultExts = []string{
".pdf", ".doc", ".docx", ".xls", ".xlsx",
".pem", ".key", ".cert",
".db", ".sqlite", ".sqlite3",
".class", // Java compiled bytecode
".jar", // JAR archives
".class",
".jar",
".war",
".ear",
}
var FrontendExts = []string{
".html", ".htm", ".gohtml", ".tmpl",
".css", ".scss", ".sass", ".less",
".js", ".jsx", ".mjs", ".cjs",
".ts", ".tsx",
".vue", ".svelte",
}
type stringSlice []string
func (s *stringSlice) String() string {
@@ -56,6 +65,7 @@ type Config struct {
InputPath string
OutputPath string
AppendMode bool
Force bool
SingleFile bool
MaxSize int64
MaxDepth int
@@ -64,6 +74,8 @@ type Config struct {
AbsRoot string
ProtonDrive bool
DrivePath string
VerticalSlices bool
SkipFrontend bool
}
func PrintUsage() {
@@ -79,6 +91,7 @@ func Parse() (*Config, error) {
input string
output string
appendMd bool
force bool
maxSize int64
maxDepth int
ignores stringSlice
@@ -86,11 +99,14 @@ func Parse() (*Config, error) {
showVer bool
protonDrive bool
drivePath string
verticalSlices bool
skipFrontend bool
)
flag.StringVar(&input, "i", "", "single file input (append mode only)")
flag.StringVar(&output, "o", "", "output markdown file path (required)")
flag.BoolVar(&appendMd, "a", false, "append to existing output file")
flag.BoolVar(&force, "f", false, "overwrite existing output file (non-append mode)")
flag.Int64Var(&maxSize, "max-size", DefaultMaxSize, "max file size in bytes")
flag.IntVar(&maxDepth, "max-depth", DefaultMaxDepth, "max directory recursion depth")
flag.Var(&ignores, "ignore", "additional ignore patterns (glob, repeatable)")
@@ -98,6 +114,8 @@ func Parse() (*Config, error) {
flag.BoolVar(&showVer, "version", false, "print version and exit")
flag.BoolVar(&protonDrive, "proton-drive", false, "upload output to Proton Drive after writing locally")
flag.StringVar(&drivePath, "drive-path", "", "full remote path on Proton Drive (requires --proton-drive)")
flag.BoolVar(&verticalSlices, "vertical-slices", false, "split output into root file + one markdown per top-level directory")
flag.BoolVar(&skipFrontend, "skip-frontend", false, "omit frontend file types (.html, .css, .js, .ts, .vue, .svelte, etc.)")
flag.Usage = PrintUsage
flag.Parse()
@@ -115,15 +133,31 @@ func Parse() (*Config, error) {
return nil, fmt.Errorf("--drive-path requires --proton-drive")
}
if verticalSlices && appendMd {
return nil, fmt.Errorf("--vertical-slices cannot be used with --append")
}
if verticalSlices && input != "" {
return nil, fmt.Errorf("--vertical-slices cannot be used with --input-file")
}
excludedExts := append(DefaultExts, exts...)
if skipFrontend {
excludedExts = append(excludedExts, FrontendExts...)
}
cfg := &Config{
OutputPath: output,
AppendMode: appendMd,
Force: force,
MaxSize: maxSize,
MaxDepth: maxDepth,
Ignores: append(DefaultIgnores, ignores...),
Extensions: append(DefaultExts, exts...),
Extensions: excludedExts,
ProtonDrive: protonDrive,
DrivePath: drivePath,
VerticalSlices: verticalSlices,
SkipFrontend: skipFrontend,
}
if input != "" {
@@ -147,8 +181,8 @@ func Parse() (*Config, error) {
cfg.AbsRoot = abs
if !appendMd {
if _, err := os.Stat(output); err == nil {
return nil, fmt.Errorf("output file %s already exists (use -a to append)", output)
if _, err := os.Stat(output); err == nil && !force {
return nil, fmt.Errorf("output file %s already exists (use -f to overwrite or -a to append)", output)
}
}
+51 -7
View File
@@ -10,12 +10,10 @@ import (
const binaryName = "proton-drive"
// Lookup returns the full path to the proton-drive binary in PATH.
func Lookup() (string, error) {
return exec.LookPath(binaryName)
}
// CheckAuth verifies that the user is authenticated by listing /my-files.
func CheckAuth() error {
binary, err := Lookup()
if err != nil {
@@ -32,11 +30,57 @@ func CheckAuth() error {
return nil
}
// Upload uploads localPath to remotePath on Proton Drive.
// remotePath is a full path including filename (e.g. /my-files/md/report.md).
// The CLI uploads to a parent directory and preserves the local filename,
// so if the remote filename differs from the local one, a temp copy is staged.
// Existing remote files are overwritten (--conflict-strategy replace).
// EnsureRemoteParent creates the parent remote directory if it doesn't exist.
func EnsureRemoteParent(remoteDir string) error {
binary, err := Lookup()
if err != nil {
return fmt.Errorf("proton-drive not found in PATH: %w", err)
}
parentDir := filepath.Dir(remoteDir)
// Try create-folder on the grandparent for the parent
grandparent := filepath.Dir(parentDir)
parentName := filepath.Base(parentDir)
cmd := exec.Command(binary, "filesystem", "create-folder", grandparent, parentName)
cmd.Stdout = nil
cmd.Stderr = nil
cmd.Run() // Ignore errors — folder may already exist
return nil
}
// UploadDir uploads an entire local directory to Proton Drive.
// The local directory's basename becomes the remote directory name.
func UploadDir(localDir, remoteDir string) error {
binary, err := Lookup()
if err != nil {
return fmt.Errorf("proton-drive not found in PATH: %w", err)
}
// Ensure parent of remote dir exists
if err := EnsureRemoteParent(remoteDir); err != nil {
return fmt.Errorf("cannot prepare remote directory: %w", err)
}
// Upload to parent — the local dir's basename creates the remote dir
remoteParent := filepath.Dir(remoteDir)
cmd := exec.Command(binary, "filesystem", "upload", localDir, remoteParent,
"--conflict-strategy", "replace")
var stderr strings.Builder
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return fmt.Errorf("upload to proton-drive failed: %w: %s",
err, strings.TrimSpace(stderr.String()))
}
return nil
}
// Upload uploads a single local file to a remote path on Proton Drive.
func Upload(localPath, remotePath string) error {
binary, err := Lookup()
if err != nil {
+354 -5
View File
@@ -4,6 +4,7 @@ import (
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"unicode/utf8"
@@ -13,6 +14,7 @@ import (
const fenceChars = "~~~"
// Write writes a single markdown file with tree and file contents.
func Write(result *walker.Result, outputPath string, appendMode bool, singleFile bool) error {
flags := os.O_CREATE | os.O_WRONLY
if appendMode {
@@ -29,20 +31,19 @@ func Write(result *walker.Result, outputPath string, appendMode bool, singleFile
var sb strings.Builder
// Determine the correct H1 path
h1Path := result.AbsRoot
if singleFile && len(result.Entries) > 0 {
h1Path = filepath.Join(result.AbsRoot, result.Entries[0].RelPath)
}
fmt.Fprintf(&sb, "# %s\n\n", h1Path)
fmt.Fprintf(&sb, "# %s [ROOT]\n\n", h1Path)
if !singleFile {
root := tree.Build(result.Entries)
rootName := filepath.Base(result.AbsRoot)
sb.WriteString("## Directory Tree\n\n")
sb.WriteString(fenceChars + "\n")
sb.WriteString(tree.Render(root, rootName))
sb.WriteString(renderTreeForLLM(root, rootName))
sb.WriteString(fenceChars + "\n\n")
}
@@ -61,8 +62,9 @@ func Write(result *walker.Result, outputPath string, appendMode bool, singleFile
}
language := extToLang(filepath.Ext(entry.RelPath))
lines := countLines(content)
fmt.Fprintf(&sb, "### ./%s\n\n", entry.RelPath)
fmt.Fprintf(&sb, "--- FILE: ./%s (%d bytes, %d lines) ---\n", entry.RelPath, entry.Size, lines)
fmt.Fprintf(&sb, "%s", fenceChars)
if language != "" {
fmt.Fprintf(&sb, "%s", language)
@@ -72,7 +74,7 @@ func Write(result *walker.Result, outputPath string, appendMode bool, singleFile
if !strings.HasSuffix(content, "\n") {
fmt.Fprintf(&sb, "\n")
}
fmt.Fprintf(&sb, "%s\n\n", fenceChars)
fmt.Fprintf(&sb, "%s\n--- END FILE ---\n\n", fenceChars)
}
}
@@ -80,6 +82,342 @@ func Write(result *walker.Result, outputPath string, appendMode bool, singleFile
return err
}
// WriteSlices generates a root markdown file plus one markdown per top-level directory.
// Returns the list of generated file paths.
func WriteSlices(result *walker.Result, outputDir, repoName string) ([]string, error) {
entries := result.Entries
if len(entries) == 0 {
return nil, fmt.Errorf("no entries to write")
}
absRoot := result.AbsRoot
rootName := filepath.Base(absRoot)
topLevelDirs := make(map[string][]walker.Entry)
rootFiles := make([]walker.Entry, 0)
for _, e := range entries {
rel := e.RelPath
parts := strings.Split(rel, string(filepath.Separator))
if len(parts) == 1 {
rootFiles = append(rootFiles, e)
} else {
firstPart := parts[0]
topLevelDirs[firstPart] = append(topLevelDirs[firstPart], e)
}
}
var generated []string
// Write root file as index_<reponame>.md
indexBasename := "index_" + repoName + ".md"
rootPath := filepath.Join(outputDir, indexBasename)
generated = append(generated, rootPath)
if err := writeRootSlice(rootPath, absRoot, rootName, repoName, rootFiles, topLevelDirs); err != nil {
return nil, fmt.Errorf("failed to write root slice: %w", err)
}
// Write each top-level directory slice into a mirroring subdirectory
var dirNames []string
for k := range topLevelDirs {
dirNames = append(dirNames, k)
}
sort.Strings(dirNames)
for _, dirName := range dirNames {
sliceName := strings.TrimSuffix(dirName, "/") + "_" + repoName + ".md"
sliceDir := filepath.Join(outputDir, dirName)
if err := os.MkdirAll(sliceDir, 0755); err != nil {
return nil, fmt.Errorf("cannot create slice directory %s: %w", sliceDir, err)
}
slicePath := filepath.Join(sliceDir, sliceName)
generated = append(generated, slicePath)
absDir := filepath.Join(absRoot, dirName)
dirEntries := topLevelDirs[dirName]
// Strip dirName prefix for tree building only
treeEntries := make([]walker.Entry, len(dirEntries))
for i, e := range dirEntries {
sepIdx := strings.Index(e.RelPath, string(filepath.Separator))
if sepIdx >= 0 {
treeEntries[i] = walker.Entry{
RelPath: e.RelPath[sepIdx+1:],
Size: e.Size,
}
} else {
treeEntries[i] = e
}
}
if err := writeDirSlice(slicePath, absDir, absRoot, dirName, treeEntries, dirEntries, indexBasename); err != nil {
return nil, fmt.Errorf("failed to write slice %s: %w", sliceName, err)
}
}
return generated, nil
}
func writeRootSlice(path, absRoot, rootName, repoName string, rootFiles []walker.Entry, topLevelDirs map[string][]walker.Entry) error {
f, err := os.Create(path)
if err != nil {
return fmt.Errorf("cannot create root file: %w", err)
}
defer f.Close()
var sb strings.Builder
fmt.Fprintf(&sb, "# %s [ROOT]\n\n", absRoot)
// Build combined tree with references
treeLines := buildTreeWithRefs(rootName, repoName, topLevelDirs, rootFiles)
sb.WriteString("## Directory Tree\n\n")
sb.WriteString(fenceChars + "\n")
sb.WriteString(treeLines)
sb.WriteString(fenceChars + "\n\n")
// Vertical slices index for LLM navigation
sb.WriteString("## Vertical Slices Index\n\n")
sb.WriteString("These files contain the full contents of each top-level directory.\n")
sb.WriteString("Read the relevant slice file for detailed source code.\n\n")
sb.WriteString("| Slice File | Directory | Files | Total Bytes |\n")
sb.WriteString("|------------|-----------|-------|-------------|\n")
var dirNames []string
for k := range topLevelDirs {
dirNames = append(dirNames, k)
}
sort.Strings(dirNames)
for _, dirName := range dirNames {
entries := topLevelDirs[dirName]
fileCount := len(entries)
totalBytes := int64(0)
for _, e := range entries {
totalBytes += e.Size
}
sliceFile := filepath.Join(dirName, strings.TrimSuffix(dirName, "/")+"_"+repoName+".md")
fmt.Fprintf(&sb, "| %s | %s/ | %d | %d |\n", sliceFile, dirName, fileCount, totalBytes)
}
sb.WriteString("\n")
if len(rootFiles) > 0 {
sb.WriteString("## Root Contents\n\n")
sort.Slice(rootFiles, func(i, j int) bool {
return rootFiles[i].RelPath < rootFiles[j].RelPath
})
for _, entry := range rootFiles {
fullPath := filepath.Join(absRoot, entry.RelPath)
content, err := readFile(fullPath)
if err != nil {
fmt.Fprintf(os.Stderr, "warning: cannot read %s: %v\n", entry.RelPath, err)
continue
}
language := extToLang(filepath.Ext(entry.RelPath))
lines := countLines(content)
fmt.Fprintf(&sb, "--- FILE: ./%s (%d bytes, %d lines) ---\n", entry.RelPath, entry.Size, lines)
fmt.Fprintf(&sb, "%s", fenceChars)
if language != "" {
fmt.Fprintf(&sb, "%s", language)
}
fmt.Fprintf(&sb, "\n")
fmt.Fprintf(&sb, "%s", content)
if !strings.HasSuffix(content, "\n") {
fmt.Fprintf(&sb, "\n")
}
fmt.Fprintf(&sb, "%s\n--- END FILE ---\n\n", fenceChars)
}
}
_, err = f.WriteString(sb.String())
return err
}
func writeDirSlice(path, absDir, absRoot, dirName string, treeEntries, fileEntries []walker.Entry, rootFileName string) error {
f, err := os.Create(path)
if err != nil {
return fmt.Errorf("cannot create slice file: %w", err)
}
defer f.Close()
var sb strings.Builder
fmt.Fprintf(&sb, "# %s [SLICE]\n\n", absDir)
fmt.Fprintf(&sb, "Root document: %s\n\n", rootFileName)
sb.WriteString("## Directory Tree\n\n")
sb.WriteString(fenceChars + "\n")
sliceTree := buildTreeFromEntries(dirName, treeEntries)
sb.WriteString(sliceTree)
sb.WriteString(fenceChars + "\n\n")
if len(fileEntries) > 0 {
sb.WriteString("## Contents\n\n")
sort.Slice(fileEntries, func(i, j int) bool {
return fileEntries[i].RelPath < fileEntries[j].RelPath
})
for _, entry := range fileEntries {
fullPath := filepath.Join(absRoot, entry.RelPath)
content, err := readFile(fullPath)
if err != nil {
fmt.Fprintf(os.Stderr, "warning: cannot read %s: %v\n", entry.RelPath, err)
continue
}
language := extToLang(filepath.Ext(entry.RelPath))
lines := countLines(content)
fmt.Fprintf(&sb, "--- FILE: ./%s (%d bytes, %d lines) ---\n", entry.RelPath, entry.Size, lines)
fmt.Fprintf(&sb, "%s", fenceChars)
if language != "" {
fmt.Fprintf(&sb, "%s", language)
}
fmt.Fprintf(&sb, "\n")
fmt.Fprintf(&sb, "%s", content)
if !strings.HasSuffix(content, "\n") {
fmt.Fprintf(&sb, "\n")
}
fmt.Fprintf(&sb, "%s\n--- END FILE ---\n\n", fenceChars)
}
}
_, err = f.WriteString(sb.String())
return err
}
func buildTreeWithRefs(rootName, repoName string, topLevelDirs map[string][]walker.Entry, rootFiles []walker.Entry) string {
var sb strings.Builder
sb.WriteString(rootName)
sb.WriteString("/\n")
// Collect and sort directory names
var dirNames []string
for k := range topLevelDirs {
dirNames = append(dirNames, k)
}
sort.Strings(dirNames)
// Sort root files
sort.Slice(rootFiles, func(i, j int) bool {
return rootFiles[i].RelPath < rootFiles[j].RelPath
})
// Merge directories and files into a single sorted list for display
type treeItem struct {
name string
isDir bool
sliceRef string
}
items := make([]treeItem, 0, len(dirNames)+len(rootFiles))
for _, d := range dirNames {
sliceRef := filepath.Join(d, strings.TrimSuffix(d, "/")+"_"+repoName+".md")
items = append(items, treeItem{
name: d,
isDir: true,
sliceRef: sliceRef,
})
}
for _, rf := range rootFiles {
items = append(items, treeItem{
name: rf.RelPath,
isDir: false,
})
}
sort.Slice(items, func(i, j int) bool {
return items[i].name < items[j].name
})
for _, item := range items {
sb.WriteString("+- ")
sb.WriteString(item.name)
if item.isDir {
sb.WriteString("/")
sb.WriteString(" -> see ")
sb.WriteString(item.sliceRef)
}
sb.WriteString("\n")
}
return sb.String()
}
func buildTreeFromEntries(rootName string, entries []walker.Entry) string {
root := &tree.Node{IsDir: true, Children: make(map[string]*tree.Node)}
for _, e := range entries {
parts := strings.Split(e.RelPath, string(filepath.Separator))
cur := root
for i, part := range parts {
isLeaf := i == len(parts)-1
if _, ok := cur.Children[part]; !ok {
node := &tree.Node{Name: part, IsDir: !isLeaf}
if !isLeaf {
node.Children = make(map[string]*tree.Node)
}
cur.Children[part] = node
}
cur = cur.Children[part]
}
}
var sb strings.Builder
sb.WriteString(rootName)
sb.WriteString("/\n")
renderNodeLLM(&sb, root, "")
return sb.String()
}
func renderTreeForLLM(root *tree.Node, rootName string) string {
var sb strings.Builder
sb.WriteString(rootName)
sb.WriteString("/\n")
renderNodeLLM(&sb, root, "")
return sb.String()
}
func renderNodeLLM(sb *strings.Builder, node *tree.Node, prefix string) {
keys := make([]string, 0, len(node.Children))
for k := range node.Children {
keys = append(keys, k)
}
sort.Strings(keys)
for i, key := range keys {
child := node.Children[key]
isLast := i == len(keys)-1
sb.WriteString(prefix)
sb.WriteString("+- ")
sb.WriteString(child.Name)
if child.IsDir {
sb.WriteString("/")
}
sb.WriteString("\n")
if child.IsDir && child.Children != nil {
newPrefix := prefix
if isLast {
newPrefix += " "
} else {
newPrefix += "| "
}
renderNodeLLM(sb, child, newPrefix)
}
}
}
func readFile(path string) (string, error) {
data, err := os.ReadFile(path)
if err != nil {
@@ -131,3 +469,14 @@ func extToLang(ext string) string {
}
return ""
}
func countLines(content string) int {
if content == "" {
return 0
}
count := strings.Count(content, "\n")
if len(content) > 0 && content[len(content)-1] != '\n' {
count++
}
return count
}