Symlinks to subdirectories in docs folder do not work, and produce unintuitive error messages

[patmantru]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

Docusaurus does not handle symlinks to directories inside the docs directory. The error message that is displayed does not help track down the problem.

This situation arose when trying to create a workflow that did a git clone on several private sibling repos into the project base directory, and then creating symlinks for the various 'docs' directories in those clones to subdirs in the project base 'docs' directory.

Reproducible demo

No response

Steps to reproduce

Steps to reproduce:

1. Create a working docusaurus environment in /base/to/docusaurus/project
2. Add a directory called 'newdir' containing .md files in the base project directory
3. ln -s /base/to/docusaurus/project/newdir /base/to/docusaurus/project/docs/newdir
4. Build
5. Get error messages

If you change the symlink to a copy command, everything works as expected

Expected behavior

I expected that the documents in the symlink'ed directories would be processed as if they were just normal subdirectories and files.

Actual behavior

Every file generated the following error:

 Error: Can't render static file for pathname "/all-docs/tagging/schemas_md/flow-definitions"
       at generateStaticFile (/Users/pmancuso/Documents/git/all-docs/node_modules/@docusaurus/core/lib/ssg/ssg.js:167:15)
       at processTicksAndRejections (node:internal/process/task_queues:105:5)
       at runNextTicks (node:internal/process/task_queues:69:3)
       at process.processImmediate (node:internal/timers:479:9)
       at async /Users/pmancuso/Documents/git/all-docs/node_modules/p-map/index.js:57:22 {
     [cause]: TypeError: Cannot read properties of undefined (reading 'id')
         at DocItem (server.bundle.js:88907:89)
         at Uc (server.bundle.js:95190:44)
         at Xc (server.bundle.js:95192:253)
         at Z (server.bundle.js:95198:89)
         at Xc (server.bundle.js:95196:231)
         at Z (server.bundle.js:95198:89)
         at Xc (server.bundle.js:95192:481)
         at Z (server.bundle.js:95198:89)
         at Vc (server.bundle.js:95190:473)
         at Xc (server.bundle.js:95192:210)
   },

Your environment

• Public source code: [private, not relevant]
• Public site URL: [private, not relevant]
• Docusaurus version used: 3.6.3
• Environment name and version (e.g. Chrome 89, Node.js 16.4): Chrome 131.0.6778.109, Node version: v23.2.0
• Operating system and version (e.g. Ubuntu 20.04.2 LTS): MacOS 15.1.1 (24B91)

Self-service

• I'd be willing to fix this bug myself.

[slorber]

As far as I remember from an old issue (#3272) we only support symlinks on the root docs folder, not subdirectories/files.

Since then, it's been a while since we haven't had any symlink issue, so the traction for this is probably low.

Curious to know more about the use case so that we figure out if it's worth supporting.

If you are assembling multiple repositories docs into one unified docs site, have you considered just copying the docs over as part of your build process, or using multiple docs plugin instances?

Your error message shows an SSG build error, but does it work in dev mode?

[patmantru]

The use case is a bunch of components for a larger overall project, each in their own private repo. The all-docs repo is a sibling to those other ones, and the idea was to gather all the docs directories from each of the components into the docs subdir of the all-docs repo, and then do one big docusaurus run to generate an overall project doc site.

The workflow runs a script that does git clone operations of all of the sibling component repos. To save time/space, I tried doing symlinks from those clones' doc directories. That didn't work (obviously), and after some digging I found mention of a symlink problem that yielded some similar behavior. I changed the symlinks to copy operations, and it is working as expected now.

Making the symlinks work certainly falls into the WIBNI category, but more importantly, at a minimum there should be a 'we don't do symlinks' error thrown instead of the current behavior.

[patmantru]

...or using multiple docs plugin instances?

That actually would be my preference. I tried this approach initially, but I couldn't get it to work since in this case the plugin is trying to access a sibling repo that is private. At some point there is a raw GET issued that does not include any auth info, and the request fails. I think the 'list the files in the dir' part worked, but the actual GET of the files was the part that was failing. This works great for the open source part of the project, since that is all public access, and those GET operations work.