Allow globby negative matches for copyWithoutTemplating

The option copyWithoutTemplating (and depracted copyWithoutRender) would
require listing all files in a path to include even if there was only
one or two files that should still be templated while skipping all
others. This works fine when just one or two files or paths that need to
be copied without being templated, however when there are many files and
only one or two should be templated, it becomes more error prone to try
and maintain a complete list and ensure others will update it correctly.

While it's possible to use separate paths to avoid this issue, it
results in subsequently needing to move files around, or use separate
template calls, this makes it more difficult to maintain a template
logically.

The existing template code already makes use of 'globby', which supports
negative matching to explicitly exclude individual paths from other
wildcard matches. The existing code would iterate over each entry
calling globby separately for each pattern passed in. This would skip
globby's ability to have negative matches remove previously matched
paths.

This change switches to pass the entire list of patterns to globby in a
single call, leaving it up to globby to parse and return the entire set
in a single operation.

Signed-off-by: Darragh Bailey <dbailey@g-p.com>

.changeset/strong-sloths-push.md

@@ -0,0 +1,5 @@
+---
+'@backstage/plugin-scaffolder-backend': minor
+---
+
+Allow using globby's negative matching with copyWithoutTemplating/copyWithoutRender. Allows including an entire subdirectory while excluding a single file so that it will still be templated instead of needing to list every other file and ensure the list is updated when new files are added.

plugins/scaffolder-backend/src/scaffolder/actions/builtin/fetch/template.test.ts

@@ -470,6 +470,57 @@ describe('fetch:template', () => {
         ),
       ).resolves.toEqual('1234');
     });
+
+    describe('with exclusion filter', () => {
+      beforeEach(async () => {
+        context = mockContext({
+          values: {
+            name: 'test-project',
+            count: 1234,
+          },
+          copyWithoutTemplating: [
+            '.unprocessed',
+            '!*/templated-process-content-${{ values.name }}.txt',
+          ],
+        });
+
+        mockFetchContents.mockImplementation(({ outputPath }) => {
+          mockDir.setContent({
+            [outputPath]: {
+              processed: {
+                'templated-content-${{ values.name }}.txt':
+                  '${{ values.count }}',
+              },
+              '.unprocessed': {
+                'templated-content-${{ values.name }}.txt':
+                  '${{ values.count }}',
+                'templated-process-content-${{ values.name }}.txt':
+                  '${{ values.count }}',
+              },
+            },
+          });
+
+          return Promise.resolve();
+        });
+
+        await action.handler(context);
+      });
+
+      it('renders path template including excluded matches in copyWithoutTemplating', async () => {
+        await expect(
+          fs.readFile(
+            `${workspacePath}/target/.unprocessed/templated-process-content-test-project.txt`,
+            'utf-8',
+          ),
+        ).resolves.toEqual('1234');
+        await expect(
+          fs.readFile(
+            `${workspacePath}/target/.unprocessed/templated-content-test-project.txt`,
+            'utf-8',
+          ),
+        ).resolves.toEqual('${{ values.count }}');
+      });
+    });
   });
 
   describe('cookiecutter compatibility mode', () => {

plugins/scaffolder-backend/src/scaffolder/actions/builtin/fetch/template.ts

@@ -207,19 +207,13 @@ export function createFetchTemplateAction(options: {
       });
 
       const nonTemplatedEntries = new Set(
-        (
-          await Promise.all(
-            (copyOnlyPatterns || []).map(pattern =>
-              globby(pattern, {
-                cwd: templateDir,
-                dot: true,
-                onlyFiles: false,
-                markDirectories: true,
-                followSymbolicLinks: false,
-              }),
-            ),
-          )
-        ).flat(),
+        await globby(copyOnlyPatterns || [], {
+          cwd: templateDir,
+          dot: true,
+          onlyFiles: false,
+          markDirectories: true,
+          followSymbolicLinks: false,
+        }),
       );
 
       // Cookiecutter prefixes all parameters in templates with