A guide to packages, setup, resolvers, utilities, and integration testing of file uploads.


Getting file uploads into our stack has been a joy. I wanted to break
down our approach in the hopes it saves someone a bit of heartache.
Here's the tools we'll be using:

- [Apollo server](https://www.apollographql.com/docs/apollo-server/)
  through [middleware](https://www.apollographql.com/docs/apollo-server/integrations/middleware/).
  - [graphql-upload](https://www.npmjs.com/package/graphql-upload)
- [Jest](https://jestjs.io/)
  - [supertest](https://www.npmjs.com/package/supertest)

For the full example of my GraphQL server setup, see the
[example project](https://github.com/lancegliser/node-express-apollo-graphql) on GitHub.

## GraphQL

I'll omit a great deal as the focus is only on upload. Your mechanism should
define its own file and ids handling. My own is based on
[ArangoDB](https://www.arangodb.com/), but that's outside this article's scope.

In this example I've removed my typical GraphQL namespacing that allows
queries and mutations as example below would show. Someday I'll get around to
an entry on that...

```graphql
mutation AddFile($file: Upload!) {
  file {
    add(file: $file) {
      _key
      displayName
    }
  }
}
```

For a basic upload, we're looking at something similar to this:

```graphql
scalar Upload
type Mutation

extend type Mutation {
  "Uploads a file to a cloud provider and returns file object"
  addFile(file: Upload!): File
}

type File implements IDocument & ICreated & IDisplayName {
  "Unique identifier for the resource across all collections"
  _id: ID
  "Unique identifier for the resource within its collection"
  _key: ID
  "Unique identifier for revision"
  _rev: String
  "ISO date time string for the time this resource was created"
  createdAt: String
  "Unique identifier for user that created this resource"
  createdBy: ID
  "The original preformatted name safe to display in any HTML context"
  displayName: String
  "A renderedable source safe to display in HTML context"
  displayImageUrl: String
  "The public url of the file"
  publicUrl: String
  "The storage provider for this file"
  storageProvider: CloudProvider
  "The storage location or file path to this file"
  bucket: String
  "The actual file name as used in storage"
  fileName: String
  "The file size in bytes"
  size: Int
  mimetype: String
  encoding: String
}

enum CloudProvider {
  GoogleCloudPlatform
  AmazonWebServices
}

interface IDocument {
  "Unique identifier for the resource across all collections"
  _id: ID
  "Unique identifier for the resource within its collection"
  _key: ID
  "Unique identifier for revision"
  _rev: String
}

interface ICreated {
  "ISO date time string for the time this resource was created"
  createdAt: String
  "Unique identifier for users that created this resource"
  createdBy: ID
}

interface IDisplayName {
  "A preformatted display name safe to display in HTML context"
  displayName: String
}
```

## Resolvers

I typically split my code into logical components. At the top level I use
an index file like this to gather all the resolvers and graphql schemas.

### `src/components/index.ts`

```ts
import { GraphQLFileLoader } from "@graphql-tools/graphql-file-loader";
import { loadSchema } from "@graphql-tools/load";
import { IResolvers } from "@graphql-tools/utils/Interfaces";
import { GraphQLSchema } from "graphql";
import { addResolversToSchema } from "@graphql-tools/schema";
import { addResolvers as addFileResolvers } from "./files/files.resolvers";
import { GraphQLUpload } from "graphql-upload";

export const getSchemaWithResolvers = async (): Promise<GraphQLSchema> => {
  // Create a schema based on all `.graphql` files in src/components
  const schema = await loadSchema("./src/**/*.graphql", {
    // load from multiple files using glob
    loaders: [new GraphQLFileLoader()],
  });
  // Prepare resolvers by adding any static packages like Upload or JSON
  const resolvers: IResolvers = {
    Query: {},
    Mutation: {},
    Upload: GraphQLUpload!,
  };

  // Augment the static resolvers with any component resolvers we've written
  addFileResolvers(resolvers);
  // Return the whole package back to the application layer so Apollo can run
  return addResolversToSchema({
    schema,
    resolvers,
  });
};
```

### `src/components/files/files.resolvers.ts`

The component resolvers can then focus on adding just their own logic,
extending the `Query` and `Mutation` objects as required. I use the
same logic for field specific resolvers as you can see commented out.

```ts
import {
  FileResolvers,
  MutationResolvers,
  Resolvers,
} from "../../generated/types";
import { addFile } from "./files.utils";

export const addResolvers = (resolvers: Resolvers): Resolvers => {
  // resolvers.Query = { ...resolvers.Query, ...query };
  resolvers.Mutation = { ...resolvers.Mutation, ...mutation };
  // resolvers.File = fileResolvers;
  return resolvers;
};

const mutation: MutationResolvers = {
  addFile: (parent, args, context) => addFile(args, context),
};

// const fileResolvers: FileResolvers = {
//   publicUrl: (parent, args, context) => {
//     if(parent.publicUrl){
//       return parent.publicUrl;
//     }
//     // TODO if requested, make the file public and return the url
//   }
// }
```

#### Where did `MutationResolvers` come from?

The autocomplete types generated by GraphQL codegen are just
so tasty it's unbelievable. Implementation time and error
counts dropped significantly once we started using them.
Huge thanks to: [The Guild's code generator](https://www.graphql-code-generator.com/). 🚀

To say "it's complicated" would be an understatement. Here's a sample of what gets
generated:

```ts
// ContextType is my own definiton passed through to the generator
export type MutationResolvers<
  ContextType = GraphQLContext,
  ParentType extends ResolversParentTypes["Mutation"] = ResolversParentTypes["Mutation"]
> = {
  /** Uploads a file to a cloud provider and returns file object */
  addFile?: Resolver<
    Maybe<ResolversTypes["File"]>,
    ParentType,
    ContextType,
    RequireFields<MutationsAddFileArgs, "file">
  >;
};

export type ResolverTypeWrapper<T> = Promise<T> | T;

export type File = ICreated &
  IDisplayName &
  IDocument & {
    __typename?: "File";
    /** Unique identifier for the resource across all collections */
    _id?: Maybe<Scalars["ID"]>;
    /** Unique identifier for the resource within its collection */
    _key?: Maybe<Scalars["ID"]>;
    /** Unique identifier for revision */
    _rev?: Maybe<Scalars["String"]>;
    /** The storage location or file path to this file */
    bucket?: Maybe<Scalars["String"]>;
    /** ISO date time string for the time this resource was created */
    createdAt?: Maybe<Scalars["String"]>;
    /** Unique identifier for user that created this resource */
    createdBy?: Maybe<Scalars["ID"]>;
    /** A renderedable source safe to display in HTML context */
    displayImageUrl?: Maybe<Scalars["String"]>;
    // ...
  };
```

The upshot of all their generated work is perfectly translated
GraphQL objects (with interfaces!) _and_ static types for your resolvers.
Many of my resolvers are written take advantage of all that typing:

```ts
const fileResolvers: FileResolvers = {
  someField: ({ id: fileId }, { limit, offset, ...args }, { user }) => {
    // ... some implementation
  },
};
```

If you're not salivating yet, I'll throw one more treat on the pile.
We use the _same_ code generation in our
[React app](https://github.com/lancegliser/react-apollo)
side generating our queries and mutations to get automatic state
management and typings:

```ts
const [addFile, { loading, error, data }] = useAddFileMutation({});
const onClick = async () => {
  await addFile({
    variables: {
      file: null, // your state variable
    },
  });
};
```

## Utilities

Apollo server will have handled most of the requirements up to now
based on just the GraphQL spec and resolvers we added. Custom bits happen in
our utilities. Your own file storage needs are going to be different. I'll
show just the basics of getting where you could store and a return here.

### `src/components/files/files.utils.ts`

```ts
import { ApolloError } from "apollo-server-express";
import { GraphQLContext } from "../context";
import fs from "fs";
import path from "path";
import {
  CloudProvider,
  File,
  MutationsAddFileArgs,
} from "../../generated/types";

const defaultProvider: CloudProvider = CloudProvider.Aws;
const bucket = "uploads";

export const addFile = async (
  // { file: Upload } as defined by graphql-upload
  args: MutationsAddFileArgs,
  // Logging, repositories, and user details
  context: GraphQLContext
): Promise<File> => {
  // Wait for the capacitor upload to complete
  const file = await args.file;

  // When a file is uploaded to the API, the extension is .tmp.
  // Before we can move the file along we need restore the previous name.
  const sourcePathComponents: string[] = file.capacitor._path
    .split(path.sep)
    .slice(0, -1);
  const targetPath = [...sourcePathComponents, file.filename].join(path.sep);
  fs.renameSync(file.capacitor._path, targetPath);

  // TODO your own storage logic on the file at targetPath

  return {
    bucket,
    displayName: file.filename,
    encoding: file.encoding,
    fileName: file.filename,
    mimetype: file.mimetype,
    size: file.capacitor.size,
    storageProvider: provider,
  };
};
```

## Testing

This example will draw slightly on my previous entry about using
[context functions](https://fragmentedthought.com/blog/2021/using-layered-context-functions-to-share-objects-for-jest-integration-testing#context-functions)
to share state between tests.

I've offloaded much of the `describe` and `it` logic, as we're only interested
in how to take a static file and use `supertest` to run an integration test
from all the way outside the app through the resolvers, db, and back.

### `src/components/files/files.test.ts`

```ts
import path from "path";
import { print } from "graphql";
import { gql } from "apollo-server-express";
import supertest from "supertest";
import { File, MutationsAddFileArgs } from "../../generated/types";

describe("files", () => {
  describe("resolvers", () => {
    it("should add file", async () => {
      await withFileContext(async ({ file }) => {
        expect(file.storageProvider).toBeTruthy();
        expect(file.bucket).toBeTruthy();
        expect(file.fileName).toBeTruthy();
      });
    });
  });
});

const fileName = "static.text";
const filePath = path.resolve(`./tests/${fileName}`);

let file: File | undefined;
type TWithFileContext = (
  context: ResolverContext & {
    file: File;
  }
) => Promise<void>;
const withFileContext = async (fn: TWithFileContext) => {
  await withResolverContext(async (context) => {
    if (!file) {
      // In this case the "file" needs a full path to upload
      file = await resolverCreateFile(context, { file: filePath });
      expect(file.fileName).toBe(fileName);
    }

    await fn({ ...context, file });
  });
};

export const resolverCreateFile = async (
  // Application, authentication token
  context: ResolverContext,
  variables: MutationsAddFileArgs
): Promise<File> => {
  // Use `gql` to create an AST tree your IDE can validate against your schemas
  // Then use `print` to change that schema into the required query string
  const query = print(gql`
    mutation AddFile($file: Upload!) {
      addFile(file: $file) {
        _id
        displayName
        storageProvider
        bucket
        fileName
      }
    }
  `);

  // We need to produce a multi-part request that's distinct from the standard graphql request.
  // It looks like the below. the -------#### are form post generated boundaries
  // The "operations" field defines the things we'll be running
  // The "map" defines what to substitute in during execution

  // Start a request, we're going to need to attach to it
  const request = supertest(context.application)
    .post(GRAPHQL_URI)
    .set("authorization", context.authorization)
    // Goal: {"operationName":"AddFile","variables":{"file":null},"query":"mutation AddFile($file: ....
    .field(
      "operations",
      JSON.stringify({
        operationName: "AddFile",
        variables: { file: null }, // This null value is substituted below
        query,
      })
    )
    // Goal: {"1":["variables.file"]}
    .field(
      "map",
      JSON.stringify({
        1: Object.keys(variables).map((key) => `variables.${key}`),
      })
    );

  // Attach the files and fields
  Object.values(variables).forEach((value, i) => {
    if (contentType(path.extname(value))) {
      request.attach(`${i + 1}`, value);
    } else {
      request.field(`${i + 1}`, value);
    }
  });

  // Send the request
  const response = await request;
  expectGraphQLSuccessResponse(response);

  return response.body.data.ontologies.addFile;
};
```

Hopefully this helps someone out there. If you liked it or have a question, please send me a
[tweet](https://twitter.com/intent/tweet?text=%40lancegliser).


GraphQL file upload mutations


A habit I developed a while back was learning to clean up after myself. Deal with
a legacy database that you can't just blow away and you'll come to realize how important
that is. We can, and _should_ [mock](https://jestjs.io/docs/mock-functions) away some of
those concerns when dealing with external systems. When unit testing, there is often
nothing to clean up anyway, or it's simple.

The technique I'll show here today is not for those use cases. It's designed
to help you with a full CRUD cycle through API graphql query and mutation, resolvers,
utils, and eventual database. Real data gets created, updated, processed, and removed.
Yes, I know that might violate some imagined laws of testing. Guess what,
it works really, really well. When working with Proof of Concept ideas,
a full suite of unit tests is overkill. Write one once a bug is identified,
or the logic can't be factored under reasonable
[cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity)
limits. Unit tests work, but they say nothing about if the system
is working on the whole.

Let's get to it.

## Overview

Today's tests will be written in [Jest](https://jestjs.io/). The
layering of context functions would apply as well in many other tools. Jest
will be able to run through `npm test`, or your own built in IDE controls
with full debugger breakpoint control. Our API will be an
[`apollo-server-express`](https://www.apollographql.com/docs/apollo-server/integrations/middleware/#apollo-server-express)
running GraphQL. We'll hand waive authorization, as that will be specific
to your implementation.

Today's task will be about manipulating an abstract `Group` that can take
generic `GroupMember` (not even users, just objects).

### `groups.graphql`

For specific details of what we'll be testing, you're welcome to review
the schema:

```graphql
extend type Query {
  "Returns all groups available to this users"
  groups: [Group!]!
  group(id: ID!): Group!
}

extend type Mutation {
  saveGroup(group: GroupInput!): Group!
  deleteGroup(id: ID!): Group
  "Adds a member to a group, based on the target selection, and returns the membership and a structure edge if created"
  addMemberToGroup(memberId: ID!, groupId: ID!): [GroupEdge!]!
  "Removes a member from a group, based on the target selection, and returns edges removed"
  removeMemberFromGroup(memberId: ID!, groupId: ID!): [GroupEdge!]!
}

type Group {
  "Unique identifier for the resource across all collections"
  id: ID
  "A preformatted display name safe to display in HTML context"
  displayName: String!
  "ISO date time String for the time this resource was created"
  createdAt: String
  "Unique identifier for users that created this resource"
  createdBy: ID
  "ISO date time String for the time this resource was created"
  updatedAt: String
  "Unique identifier for users that created this resource"
  updatedBy: ID
  "Short description"
  description: String
  "The documents that are a part of the group"
  members: [GroupMember!]
}

input GroupInput {
  "Unique identifier for the resource within its collection"
  id: ID
  "A preformatted name safe to display in any HTML context"
  displayName: String!
  "Short description"
  description: String
}

"A common set of interfaces implemented by all members allowing generic handling"
type GroupMember {
  "Unique identifier for the resource across all collections"
  id: ID
  "A preformatted display name safe to display in HTML context"
  displayName: String!
  "ISO date time String for the time this resource was created"
  createdAt: String
  "Unique identifier for users that created this resource"
  createdBy: ID
  "ISO date time String for the time this resource was created"
  updatedAt: String
  "Unique identifier for users that created this resource"
  updatedBy: ID
}
```

## Integration testing

For those not familiar with Jest, Mocha, Chai, Cypress, the syntax is simple enough:

```ts
// Create an arbitraty group of tests relating to some concept
describe("groups", () => {
  // Logic you can run before our tests in this group
  beforeAll(() => {});

  // Prepare a specific piece of fucntionality you want to know is working
  it("should create a group", async () => {
    // Implement creation and expectation logic
  });

  // Logic you can run after our tests in this group
  beforeAll(() => {});
});
```

### Context functions

I use context functions as a means of organizing code. You could do much
of this inline, using `beforeAll`, but it gets complicated as tests
require additional specifics for operations. Using context functions
allow you separate test specific requirements from `describe` group
concerns.

The goal will become to write tests that look like this. We eliminate
the `beforeAll` setup requirements, and pull just the pieces we need
from context during testing.

```ts
describe("Groups", () => {
  it("should create", async (done) => {
    await withGroupContext(async ({ group }) => {
      expectGroup(group); // Specifics are not important for this example
    });
  });
});
```

What exactly does `withGroupContext` offer?

- A `let` variable we can use for storage to ensure we can cache results.
- A `Group` created through a utility function that requires `ResolverContext`.

```ts
// A helper context function to ensure all tests have access to the Group.
// This function can be placed anywhere, such as the end of the relevant
// `describe`, or at the bottom.

// One important bit, do not export `withGroupContext`. It relies
// on `let group`. It works well in one file,  but `npm test` will suffer
// polution.
let group: Group | undefined;
type TWithGroupContext = (
  context: ResolverContext & {
    group: Group;
  }
) => Promise<void>;
const withGroupContext = async (fn: TWithGroupContext) => {
  await withResolverContext(async (context) => {
    if (!group) {
      group = await resolverSaveGroup(context, {
        group: {
          description: "Generated through resolver tests",
          displayName: new Date().toLocaleString(),
        },
      });
    }

    await fn({ ...context, group });
  });
};
```

And `withResolverContext`, what is that?

```ts
import { Application } from "express";

export interface ResolverContext {
  // A set of system contexts, in case I need to alter the DB directly.
  context: SystemContext;
  // An initialized application we can hand off to supertest
  application: Application;
  // An auth token to include on requests
  authorization: string;
}
// Some local cache variables
let application: Application;
let context: SystemContext;
// Definition of what we'll offer
type TWithResolverContext = (context: ResolverContext) => Promise<void>;
// Implementation
export const withResolverContext = async (
  fn: TWithResolverContext
): Promise<void> => {
  if (!application) {
    application = await getApp();
  }
  if (!context) {
    context = await getSystemContext();
  }
  // This will be up to your requirements
  const authorization = await getAuthorization();

  await fn({
    context,
    application,
    authorization,
  });
};

let app: Application;
export const getApp = async (): Promise<Application> => {
  if (app) {
    return app;
  }

  // Whatever your application start up logic is, so long as it's async.
  // I started using async start up apps so database and message queue
  // schemas could run first. The pay off has been huge.
  app = await getApplication();
  return app;
};
```

These examples are super basic. A resolver context function that provides nothing more than the app,
authorization, and some utilities. Layered on that is a single group we created. It demonstrates
how I like to separate concerns and implementation into small reusable pieces. How that gets used is
next.

### Integration testing example

Here's the result of those context functions I've omitted past the basic concepts to focus on usage.
The lovely parts about writing this way to me are:

- You can fire any single test, and it always has all the data required.
- In a debugger breakpoint, you can stop the test and data will be left in the system.
- If you allow it to finish, all data will be cleared, even if you don't run the `delete` test.
  Pay special attention to the `delete` and `afterAll` handling at the end for how this works.

You'll notice a lot of types being imported from `../../generated/types`. We're wired up
to [GraphQL Code Generator](https://www.graphql-code-generator.com/) which produces
Typescript based on our earlier `groups.graphql` file. For details see my
Apollo server Express GraphQL API [example](https://github.com/lancegliser/node-express-apollo-graphql)
project.

```ts
import { print } from "graphql";
import gql from "graphql-tag";
import {
  Group,
  GroupMember,
  MutationAddMemberToGroupArgs,
  MutationDeleteGroupArgs,
  MutationRemoveMemberFromGroupArgs,
  MutationSaveGroupArgs,
  QueryGroupArgs,
} from "../../generated/types";
import {
  ResolverContext,
  expectGraphQLSuccessResponse,
  withResolverContext,
} from "../../../tests/graphql.utils";
import { GRAPHQL_URI } from "../../constants";
import supertest = require("supertest");

describe("Groups", () => {
  it("should create", async () => {
    await withGroupContext(async ({ group }) => {
      expectGroup(group);
    });
  });

  it("should update", async () => {
    await withGroupContext(async (context) => {
      const { group } = context;
      if (!group.id) {
        throw new Error(`Group.id is undefined`);
      }

      const { id, description } = group;
      const newName = `Test ${new Date().toLocaleString()}`;
      const result = await resolverSaveGroup(context, {
        group: {
          id,
          description,
          displayName: newName,
        },
      });
      expectGroup(result);
      expect(result.displayName).toBe(newName);
    });
  });

  // I've left this full example call and response example in case you want
  // to start writing your own API integration tests using supertest.
  it("should get", async () => {
    await withGroupContext(async ({ application, authorization, group }) => {
      if (!group.id) {
        throw new Error(`Group.id is undefined`);
      }

      const queryData: {
        query: string;
        variables: QueryGroupArgs;
      } = {
        // `gql` provides IDE support by turning it into AST tree allowing
        // error checking and refactoring support. `print` then turns that
        // AST tree back into a string we can use in our API call.
        query: print(gql`
          query Group($key: ID!) {
            group(key: $key) {
              id
              displayName
              createdAt
              createdBy
              updatedAt
              updatedBy
              members {
                id
                displayName
              }
            }
          }
        `),
        variables: { key: group.id },
      };

      // Super test uses our definition of the application to spin up
      // on a random port and allow external API testing.
      const response = await supertest(application)
        .post(GRAPHQL_URI)
        .set("authorization", authorization)
        .send(queryData);
      expectGraphQLSuccessResponse(response);
      const result = response.body.data.group;
      expectGroup(result);
      expect(result.id).toBe(group.id);
    });
  });

  it("should get groups for the user", async () => {
    await withGroupContext(async ({ application, authorization, group }) => {
      // ...
    });
  });

  describe("membership", () => {
    it("should add members", async () => {
      await withGroupsContext(
        async ({ groupA, groupB, groupBEdges, groupC, groupCEdges }) => {
          // ...
        }
      );
    });

    it("should remove membership", async () => {
      await withGroupsContext(
        async ({ application, authorization, groupA, groupC, groupCEdges }) => {
          // ...
        }
      );
    });

    // Remember those clean up functions I promised?
    // We can hook the `afterAll` up to our context to help tear things down.
    afterAll(async () => {
      await withGroupsContext(async (context) => {
        const { groupB, groupC } = context;
        await resolverDeleteGroup(context, { id: groupB.id! });
        await resolverDeleteGroup(context, { id: groupC.id! });
      });
    });

    // Here's an example of a context function at the end of a `describe`
    // block. I personally find it hard to deal with these long context
    // functions inside `describe` and `it` blocks. I like them to be
    // simple to read, and abstract out nearly everything I can that isn't
    // expectations normally. But, it does make it impossible for other
    // developers to reuse these, preventing the context polution I
    // warned you about, and some people prefer code colocated as closely
    // as possible.
    let groupB: Group | undefined;
    let groupBEdges: GroupEdge[] | undefined;
    let groupC: Group | undefined;
    let groupCEdges: GroupEdge[] | undefined;

    // A helper context function to ensure all tests have access to the
    // some groups with some structure.
    type TWithGroupContext = (
      context: ResolverContext & {
        groupA: Group;
        groupB: Group;
        groupBEdges: GroupEdge[];
        groupC: Group;
        groupCEdges: GroupEdge[];
      }
    ) => Promise<void>;
    const withGroupsContext = async (fn: TWithGroupContext) => {
      // We'll layer on top of the existing group to create two more
      // and make them related to the first.
      await withGroupContext(async (context) => {
        const { group: groupA } = context;

        // Setup B as a member
        if (!groupB) {
          groupB = await resolverSaveGroup(context, {
            group: {
              description: "Generated through resolver tests",
              displayName: `B - ${new Date().toLocaleString()}`,
            },
          });
        }
        if (!groupBEdges) {
          groupBEdges = await resolverAddMemberToGroup(context, {
            memberId: groupB._id!,
            targetSelection: {
              target: GroupTargetSelectionTarget.Existing,
              groupKey: groupA.id,
            },
          });
        }
        // Setup C as a member, and a child of B
        if (!groupC) {
          groupC = await resolverSaveGroup(context, {
            group: {
              description: "Generated through resolver tests",
              displayName: `C - ${new Date().toLocaleString()}`,
            },
          });
        }
        if (!groupCEdges) {
          groupCEdges = await resolverAddMemberToGroup(context, {
            memberId: groupC._id!,
            targetSelection: {
              target: GroupTargetSelectionTarget.Existing,
              groupKey: groupA.id,
              parentId: groupB._id,
            },
          });
        }

        await fn({
          ...context,
          groupA,
          groupB,
          groupBEdges,
          groupC,
          groupCEdges,
        });
      });
    };
  });

  // Back outside of the `describe("membership")`, we need to clean up
  // our remaining group. Which should be tested all on it's own to make
  // sure it works.
  it("should delete", async () => {
    await withGroupContext(async (context) => {
      const { group } = context;
      if (!group.id) {
        throw new Error(`Group.id is undefined`);
      }

      const result = await resolverDeleteGroup(context, {
        key: group.id!,
      });
      expectGroup(result);
    });
  });

  // We can hook in through `afterAll` to ensure that even if the
  // delete test isn't run we still clean up after ourselves.
  afterAll(async () => {
    await withGroupContext(async (context) => {
      const { group } = context;
      // We just have to try / catch
      try {
        await resolverDeleteGroup(context, { key: group.id! });
      } catch (error) {
        // Because we would expect there to be "group not found" errors
        throwUnexpectedDeleteErrors(error);
      }
    });
  });
});

// A helper context function to ensure all tests have access to a Group
let group: Group | undefined;
type TWithGroupContext = (
  context: ResolverContext & {
    group: Group;
  }
) => Promise<void>;
const withGroupContext = async (fn: TWithGroupContext) => {
  await withResolverContext(async (context) => {
    if (!group) {
      group = await resolverSaveGroup(context, {
        group: {
          description: "Generated through resolver tests",
          displayName: new Date().toLocaleString(),
        },
      });
    }

    await fn({ ...context, group });
  });
};
```

What do you think? Is it heresy, to create local scope cache and pass it around for testing?
Do the layered context functions provide more simple abstractions for maintenance? Do
you prefer to see the context functions outside of the `describe` blocks?

Send me a
[tweet](https://twitter.com/intent/tweet?text=%40lancegliser%20RE%3A%20Layered%20context%20functions%20for%20testing.%20I...).

> I like it when people say hi.

Yes Aaron Mahnke. I stole your line. It's simple, lovely, and true.
Thanks for all the [stories](https://www.lorepodcast.com/) 👻📖.


Using layered context functions to share objects for Jest integration testing


We came on an interesting problem this week. The services team
needed to define the fields that could be indexed, and how,
dynamically based on customer supplied ontologies. None of us
can know what can be searched, or how, about what. We're _still_
going to produce an advanced search that gives field specific
search controls. Woo! Sounds like fun. Credit goes to
[Liz Townsend](https://twitter.com/liztownmo) for our
original approach, which I've expanded on here.

If you like you can skip the specifics, and go straight to
the [React implementation](#react-implementation).

## API contract

After some basic conversations we came to the agreement
they could craft an API capable of sending field schemas.
The basic idea was to send an array of fields, and we'd send back
an array of values paired with those field ids. We landed
on this schema. Some liberties have been taken to abstract from the
real usage. Perhaps the full schema is overkill for understanding the problem,
hopefully someone out there gets enjoys it just the same.

### GraphQL schema

```graphql
# See https://github.com/taion/graphql-type-json
# It allows us to send anything we need back and forth without validation
scalar JSON

extend type Query {
  "Returns the fields a user may search by"
  searchFields: [FieldSchema!]
  "Returns objects that match your search criteria"
  search(
    "A string of text from the user to be used in full text mode against all indexed data"
    query: String
    "A flexible set of values based on the searchFields used to make more precise searching"
    filters: [FieldSchemaInput!]
    "The number of records in this set"
    limit: Int
    "The index of the first item in this result set from the larger collection"
    offset: Int
  ): ObjectsPagedResponse
}

enum DataType {
  String
  Number
  Date
  Coordinates
}

"Defines schema meta data about a field"
type FieldSchema {
  type: DataType!
  "An id used to match up to the values : i.e. AWSRegion"
  fieldId: ID!
  "A preformatted display name safe to display in HTML context : AWS Region"
  displayName: String!
  "Defines if this field is required"
  required: Boolean
  "A set of options to be used for limited choice options  - Ex: [us-east-1, us-west]"
  options: [FieldSchemaOption!]
  "Allows the selection of multiple options"
  multiple: Boolean
}

type FieldSchemaOption {
  value: JSON!
  "A preformatted display name safe to display in HTML context : AWS Region"
  displayName: String!
}

input FieldSchemaInput {
  "An id used to match up to the values : i.e. AWSRegion"
  fieldId: ID!
  "The current value of this schema"
  value: JSON!
}
```

To provide a bit more clarity, I've generated some examples of usages:

#### Example `searchFields` response

```json
[
  {
    "type": "String",
    "fieldId": "document.classification.number",
    "displayName": "Classification number",
    "required": null,
    "options": null,
    "multiple": null
  },
  {
    "type": "String",
    "fieldId": "document.revision.comment",
    "displayName": "Revision comment",
    "required": null,
    "options": null,
    "multiple": null
  }
]
```

#### Example `search` query

```json
{
  "operationName": "Search",
  "variables": {
    "query": "ownership",
    "filters": [
      {
        "fieldId": "document.classification.number",
        "value": "1234"
      },
      {
        "fieldId": "document.revision.comment",
        "value": "Jane"
      }
    ]
  },
  "query": "query Search($query: String, $filters: [FieldSchemaInput!], $limit: Int, $offset: Int) {\n  search(query: $query, filters: $filters, limit: $limit, offset: $offset) {\n      limit\n      offset\n      total\n      items { ... }\n}\n"
}
```

## React implementation

This will be broken into two parts. My goal in implementing this functionality was
to abstract away from the display component everything we could about state management.
For `EntityField`, `value`, and `setValue` the props should be dirt simple, just the
return of `const [value, useValue] = useState("")`.

At a high level the separate of concerns would then look like this:

`SchemaFields.tsx` would provide props that define the shape, and value handling
like a basic input field:

```ts
export interface SchemaFieldProps<T = string> {
  value: T;
  setValue: Dispatch<SetStateAction<T>>;
  schema: FieldSchema;
  fieldProps?: TextFieldProps;
  /** Provides a method to override the simple MenuItem rendering of the select if required */
  children?: ReactNode | undefined;
}
```

`SearchForm.tsx` would then responsible for keeping state that could provide
state that could match the `value` and `setValue` handling by id. It can
be expressed in Typescript this way:

```ts
type FieldValue = string;
type FieldsState = Record<
  FieldSchema["fieldId"], // Just a string
  [
    // The value property
    FieldValue | undefined,
    // The setValue property
    Dispatch<SetStateAction<FieldValue | undefined>>
  ]
>;
```

With `FieldsState` in mind, we can declare our state this way.

```ts
// Start with an empty object, we'll have to remap our API results into it
// after the component mounts.
const [fieldsState, setFieldsState] = useState < FieldsState > {};
```

Through whatever mechanism you make your API call, the resulting `EntityFields`
array must be used to update our `fieldsState`:

```ts
const fields =
  // Fields, no fields? Who cares, make an array and we'll reduce it
  (fieldSchemasQuery.data?.searchFields ?? [])
    // Each pass of the reducer we'll add a new object key consisting
    // of a `value` and function handles like `setValue`.
    .reduce((fields, schema) => {
      fields[schema.fieldId] = [
        // Just our initial value, in this example it's safe for me to use a string.
        // Your miliage will vary, and might need more complex defaults like:
        // schema.type === "Number" ? 0 : "" handling
        "",
        // This was the yummy part for me. We have to return a setState mimic.
        // (Shout out to Neven Iliev's mimic! I ❤️ your murder hobo. It's delightful guilty pleasure.)
        // We'll need to take a value (which could be the actual value, or a function)
        (value) =>
          // We're going return our own function wrapping the behavior of the
          // standard `setValue` behavior.
          setFieldsState((prevState) => {
            // We can get the internal `setValue` behavior from state
            const [, setValue] = prevState[schema.fieldId];
            // At the time, I didn't need to support dispatch `setValue` internals.
            // This logic would expand if you did.
            if (typeof value === "function") {
              throw new Error(
                "This pattern does not currently support functions"
              );
            }

            return {
              ...prevState,
              // We can append the new value, along with our
              // wrapped `setValue` to added to our dynamic state.
              [schema.fieldId]: [value, setValue],
            };
          }),
      ];
      return fields;
    }, {} as FieldsState);
setFieldsState(fields);
```

At that point you'll be able to loop your new dynamic `fieldsState` to:

Render the entity fields:

```jsx
<>
  {fieldSchemasQuery.data?.searchFields
    // The filter is just to solve TS issues caused
    // by Maybe<string> | undefined nature of SchemaField.fieldId
    ?.filter((schema) => fieldsState[schema.fieldId])
    ?.map((schema) => {
      const { fieldId, type } = schema;
      // Here's that usage we were hoping for.
      // Dirt simple, and delicious.
      const [value, setValue] = fieldsState[fieldId];
      return (
        <FieldContainer key={fieldId}>
          <SchemaField
            schema={{
              ...schema,
              type: getDisplayTypeFromDataType(type),
            }}
            value={value}
            setValue={setValue}
            fieldProps={{
              type: "search",
              inputProps: {
                name: fieldId,
              },
            }}
          />
        </FieldContainer>
      );
    })}
</>
```

Collect the field values and submit to search:

```ts
const onSubmit: FormEventHandler = (event) => {
  event.preventDefault();

  // However your search works, as long as it accepts
  // an array of { value: unknown, fieldId: string} like filters
  // in this example
  navigateToSearch(navigate, {
    // A class state usage
    query: query,
    // Our dynamic fields
    filters: Object.keys(fieldsState)
      .filter((fieldId) => Boolean(fieldsState[fieldId][0]))
      .map(
        (fieldId) => ({
          fieldId,
          value: fieldsState[fieldId][0],
        }),
        {} as InputMaybe<Array<FieldSchemaInput>>
      ),
  });
};
```

I've included some examples of the a `SearchForm` and `EntityField` I used below.
As with most of my current work, it relies on [Material UI 4.x](https://v4.mui.com/).

### `SearchForm.tsx` example

```jsx
import React, {
  Dispatch,
  FormEventHandler,
  SetStateAction,
  useEffect,
  useState,
} from "react";
import { useNavigate } from "react-router-dom";
import { navigateToSearch } from "../../../../utils/search";
import {
  DataType,
  FieldSchema,
  useSearchFieldsQuery,
} from "../../../../generated/types";
import { Box, Button, TextField } from "@material-ui/core";
import { Skeleton } from "@material-ui/lab";
import SchemaField, {
  SchemaFieldProps,
} from "./SchemaField/SchemaField";

interface Props {}
const SearchForm: React.FunctionComponent<Props> = () => {
  const navigate = useNavigate();

  const [query, setQuery] = useState("");
  const fieldSchemasQuery = useSearchFieldsQuery();

  const [fieldsState, setFieldsState] = useState<FieldsState>({});
  useEffect(() => {
    const fields = (
      fieldSchemasQuery.data?.searchFields ?? []
    ).reduce((fields, schema) => {
      fields[schema.fieldId] = [
        "",
        (value) =>
          setFieldsState((prevState) => {
            const [, setValue] = prevState[schema.fieldId];
            if (typeof value === "function") {
              throw new Error(
                "This pattern does not currently support functions"
              );
            }

            return {
              ...prevState,
              [schema.fieldId]: [value, setValue],
            };
          }),
      ];
      return fields;
    }, {} as FieldsState);
    setFieldsState(fields);
  }, [fieldSchemasQuery.data]);

  const onSubmit: FormEventHandler = (event) => {
    event.preventDefault();

    navigateToSearch(navigate, {
      query: query,
      filters: Object.keys(fieldsState)
        .filter((fieldId) => Boolean(fieldsState[fieldId][0]))
        .map(
          (fieldId) => ({
            fieldId,
            value: fieldsState[fieldId][0],
          }),
          {} as InputMaybe<Array<FieldSchemaInput>>
        ),
    });

    reset();
  };

  const reset = () => {
    setQuery("");
    setFieldsState((prevState) =>
      Object.keys(prevState).reduce((fieldsState, fieldId) => {
        const [, setValue] = prevState[fieldId];
        fieldsState[fieldId] = ["", setValue];
        return fieldsState;
      }, {} as FieldsState)
    );
  };

  return (
    <form onSubmit={onSubmit}>
      <TextField
        autoFocus
        value={query}
        onChange={(event) => setQuery(event.target.value)}
      />

      {fieldSchemasQuery.loading &&
        new Array(3).fill(0).map((_, index) => (
          <FieldContainer key={index}>
            <Skeleton variant="rect" height={60} />
          </FieldContainer>
        ))}
      {fieldSchemasQuery.data?.searchFields
        ?.filter((schema) => fieldsState[schema.fieldId])
        ?.map((schema) => {
          const { fieldId, type } = schema;
          const [value, setValue] = fieldsState[fieldId];
          return (
            <FieldContainer key={fieldId}>
              <SchemaField
                schema={{
                  ...schema,
                  type: getDisplayTypeFromDataType(type),
                }}
                value={value}
                setValue={setValue}
                fieldProps={{
                  type: "search",
                  inputProps: {
                    name: fieldId,
                  },
                }}
              />
            </FieldContainer>
          );
        })}

      <Button variant={"contained"} color={"primary"} type={"submit"}>
        Search
      </Button>
    </form>
  );
};
export default SearchForm;

const FieldContainer: React.FunctionComponent = ({ children }) => (
  <Box my={2}>{children}</Box>
);

const getDisplayTypeFromDataType = (
  type: FieldSchema["type"]
): SchemaFieldProps["schema"]["type"] => {
  switch (type) {
    case DataType.String:
      return "string";
    case DataType.Number:
      return "number";
    default:
      throw new Error(`Unhandled FieldSchema["type"]: ${type}`);
  }
};

type FieldValue = string;
type FieldsState = Record<
  FieldSchema["fieldId"],
  [FieldValue | undefined, Dispatch<SetStateAction<FieldValue | undefined>>]
>;
```

#### `EntityField.tsx` example

_Disclaimer: This component uses
[React function components with Typescript Generics](/blog/2021/typescript-generics-type-variables-in-react-function-components).
Make sure to look up that syntax if you're new to it._

```jsx
import React, {
  Dispatch,
  ReactElement,
  ReactNode,
  SetStateAction,
} from "react";
import { MenuItem } from "@material-ui/core";
import { Maybe } from "graphql/jsutils/Maybe";
import Select, { getSelectChangeHandler } from "../Select/Select";
import TextField, {
  getTextFieldChangeHandler,
  TextFieldProps,
} from "../TextField/TextField";

export interface SchemaFieldProps<T = string> {
  value: T;
  setValue: Dispatch<SetStateAction<T>>;
  schema: FieldSchema;
  fieldProps?: TextFieldProps;
  /** Provides a method to override the simple MenuItem rendering of the select if required */
  children?: ReactNode | undefined;
}
const SchemaField = <T,>(
  props: SchemaFieldProps<T>
): ReactElement<any, any> => {
  const { schema } = props;
  return (
    <React.Fragment key={schema.fieldId}>
      {!schema.options?.length ? (
        <SchemaTextField<T> {...props} />
      ) : (
        <SchemaSelectField<T> {...props} />
      )}
    </React.Fragment>
  );
};
export default SchemaField;

/*
 * Interior data type specific rendering. These should not be exported
 */

const SchemaTextField = <T,>({
  value,
  setValue,
  schema,
  fieldProps = {},
}: SchemaFieldProps<T>): ReactElement<any, any> => {
  fieldProps.inputProps = fieldProps.inputProps || {};
  switch (schema.type) {
    case "number":
      fieldProps.inputProps.inputMode = fieldProps.inputMode || "numeric";
      break;
  }

  return (
    <TextField
      label={schema.displayName}
      required={schema?.required || false}
      value={value}
      onChange={getTextFieldChangeHandler(setValue)}
      {...fieldProps}
    />
  );
};

const SchemaSelectField = <T,>({
  value,
  setValue,
  schema,
  fieldProps,
  children,
}: SchemaFieldProps<T>): ReactElement<any, any> => {
  return (
    <Select
      label={schema.displayName}
      required={schema?.required || false}
      value={value}
      onChange={getSelectChangeHandler(setValue)}
      {...fieldProps}
    >
      {children
        ? children
        : (schema.options || []).map((option) => (
            <MenuItem key={option.value} value={option.value || undefined}>
              {option.displayName || option.value}
            </MenuItem>
          ))}
    </Select>
  );
};

/** Defines schema meta data about a field */
export type FieldSchema = {
  /** A preformatted display name safe to display in HTML context : AWS Region */
  displayName: string;
  /** An id used to match up to the values : i.e. AWSRegion */
  fieldId: string | number;
  /** Allows the selection of multiple options */
  multiple?: Maybe<boolean>;
  /** A set of options to be used for limited choice options  - Ex: [us-east-1, us-west] */
  options?: Maybe<Array<FieldSchemaOption>>;
  /** Defines if this field is required */
  required?: Maybe<boolean>;
  type: "number" | "string"; // TODO "coordinates" | "date"
};

export type FieldSchemaOption = {
  /** A preformatted display name safe to display in HTML context : US East 1A */
  displayName?: React.ReactNode;
  /** The value of this option: us-east-1a */
  value: string | number | undefined | null;
};
```

🎄 Merry Christmas 🎄 ya'll.


Creating dynamic state using state in React


If you've found this entry, you likely already know why you'd want a
[Generic type variable](https://www.typescriptlang.org/docs/handbook/2/generics.html#working-with-generic-type-variables).
Good on you! Enjoy the yummy syntax. If you're new to the concept, I'd still
like you to come at this topic from a code approach. Trust me, the actual syntax
difference is minimal. Hopefully you find it a comfort to see how similar it is.

## Creating a Generic React.ReactFunction component

Ignoring a few of my other syntax choices, I'm going to write this out
with comments inline to help explain alternatives and why I choose the options
I have.

### `GenericComponent.tsx`

```tsx
import React, { ReactElement } from "react";

// Setup a generic interface which will inform our consuming components.
export interface GenericComponentProps<T extends unknown = unknown> {
  onSelect: (event: React.MouseEvent, selected: T) => void;
  options: { value: T; content: React.ReactNode }[];
}

// The most simple version of the arrow syntax uses a "comma hack"
// const GenericComponent = <T,>(props) => {}

// This syntax works great, until you need to provide a default.
// With this, your users will be forced to supply a generic type,

// Using the extends syntax is roughly the same, but allows the assignment of a default.
const GenericComponent = <T extends unknown = unknown>({
  onSelect,
  options,
}: //
// Most of us are comfortable with the standard syntax below:
// const GenericComponent: React.Function: () => {}

// Under the covers, it's really just:
// (props: PropsWithChildren<P>, context?: any): ReactElement<any, any> | null;

// We can manually reproduce that here accepting the generic from the arrow function.
GenericComponentProps<T>): ReactElement<any, any> | null => {
  return (
    <div>
      {options.map(({ value, content }, index) => (
        <button key={index} onClick={(event) => onSelect(event, value)}>
          {content}
        </button>
      ))}
    </div>
  );
};
export default GenericComponent;
```

## Using a React.ReactFunction that accepts Generics

I've written some examples in Storybook in case you'd like to pull them down to play with.

### `GenericComponent.stories.tsx`

First a bit of Storybook boilerplate for registration:

```tsx
import GenericComponent, { GenericComponentProps } from "./GenericComponent";
import { withCardDecorators } from "../../../../../.storybook/decorators";
import { ComponentMeta, ComponentStory } from "@storybook/react";
import React, { useState } from "react";

export default {
  title: "Forms/Inputs/GenericComponent",
  component: GenericComponent,
  decorators: [...withCardDecorators],
  // I'm leaving the generic type off here on purpose.
  // The options are going to be embedded in the Template directly instead.
  // This way lets us show the effects of the Generic more easily.
  args: {} as GenericComponentProps,
  // For a real component that accepts generics you should expect to supply them
  // along with some meaninful default args to keep to enable Storybook controls.
  // args: {
  //   { content: <p>One</p>, value: 1 },
  //   { content: <p>Two</p>, value: 2 },
  //   { content: <p>North</p>, value: 3 },
  // ]} as GenericComponentProps<number>,
} as ComponentMeta<typeof GenericComponent>;
```

#### Omitting the generic

```tsx
const UnknownTemplate: ComponentStory<typeof GenericComponent> = ({
  ...props
}) => (
  <GenericComponent
    {...props}
    options={[
      // Success!
      { content: <p>One</p>, value: "one" },
      // TS2322: Type 'number' is not assignable to type 'string'.
      { content: <p>Two</p>, value: 2 },
      // Success!
      { content: <p>North</p>, value: "north" },
    ]}
    // Our selected type is not supplied
    onSelect={(event, selected) => console.info(event, selected)}
  />
);
export const Unknown = UnknownTemplate.bind({});
```

#### Using a string union type

```tsx
type CardinalDirectionType = "north" | "south" | "east" | "west";
const TypeGenericTemplate: ComponentStory<typeof GenericComponent> = ({
  ...props
}) => (
  <GenericComponent<CardinalDirectionType>
    {...props}
    options={[
      // TS2322: Type '"one"' is not assignable to type 'CardinalDirectionType'.
      { content: <p>One</p>, value: "one" },
      // TS2322: Type 'number' is not assignable to type 'CardinalDirectionType'.
      { content: <p>Two</p>, value: 2 },
      // Success!
      { content: <p>North</p>, value: "north" },
    ]}
    // Our selected is of type CardinalDirectionType
    onSelect={(event, selected) => console.info(event, selected)}
  />
);
export const TypeGeneric = TypeGenericTemplate.bind({});
```

#### Using a enum

```tsx
enum CardinalDirectionEnum {
  North = "north",
  // Yes, this is a very silly thing to do. But it revealed
  // an interesting behavior to desribe. See: value: 2.
  South = 0,
  East = "east",
  West = 1,
}
const EnumGenericTemplate: ComponentStory<typeof GenericComponent> = ({
  ...props
}) => (
  <GenericComponent<CardinalDirectionEnum>
    {...props}
    options={[
      // TS2322: Type '"one"' is not assignable to type 'CardinalDirectionEnum'.
      { content: <p>One</p>, value: "one" },
      // Shockingly, this is valid. That's because Enum could be 2, even though I didn't use it.
      // If I don't provide mixed enum values and make them all strings, we get:
      // TS2322: Type 'number' is not assignable to type 'CardinalDirectionEnum'.
      // The lesson here is to provide actual enum values as strings when possible!
      { content: <p>Two</p>, value: 2 },
      // TS2322: Type '"north"' is not assignable to type 'CardinalDirectionEnum'.
      // Not surprising. Just because the string is "north" doesn't mean it's CardinalDirectionEnum.North
      { content: <p>North</p>, value: "north" },
      // Success!
      { content: <p>North</p>, value: CardinalDirectionEnum.North },
    ]}
    // Our selected is of type CardinalDirectionType
    onSelect={(event, selected) => console.info(event, selected)}
  />
);
export const EnumGeneric = EnumGenericTemplate.bind({});
```

## Why should we write components with Generics?

Now comes the bit I promised: reasons. Sorry, the free beer is still tomorrow.

Off the top, I think it's important to point out a design choice I made this
example. I provided a default type to the generic: `<T extends unknown = unknown>`.
That's a pretty major design decision and not one you should take lightly.

The default you provide would rarely be `unknown` as you could expect more
structured types or interfaces. Use this option when you know the basic properties.

I would personally suggest that more often than not, you won't know best. That
is _why_ we are making it generic after all. My tendency is to the `<T,>` syntax.
The big selling point to this method is the automatic inspection of the first
argument consumers provide. Whatever they say it is, the rest has to match.
If that's ever a problem, they can take the power by using our generic.

The biggest win comes from those that know ahead of time they'll want strong typing.
Using a declared type like `CardinalDirectionEnum` makes defining `options.value`
autocomplete. It's also the only way we get a type for the `onSelected` value.

### Things that don't work so well

This is actually the second attempt at writing this entry. The first sent me off
from the code defeated, and a bit enraged. Let's talk for just a moment about
the basic `input`, `select`, `textarea` properties `value` and `onChange`.
Here's the component I started off with:

```tsx
import React, { Dispatch, ReactElement, SetStateAction } from "react";

export interface GenericComponentProps<T extends unknown = unknown> {
  value: T;
  setValue: Dispatch<SetStateAction<T>>;
  options: { value: T; content: React.ReactNode }[];
}

// Using the extends syntax is roughly the same, but allows the assignment of a default
const EnumSelect = <T extends unknown = unknown>({
  value,
  setValue,
  options,
}: GenericComponentProps<T>): ReactElement<any, any> => {
  return (
    <select value={value} onChange={(event) => setValue(event.target.value)}>
      {options.map(({ value, content }, index) => (
        <option key={index}>{content}</option>
      ))}
    </select>
  );
};
export default EnumSelect;
```

This thing. This maddening thing right here:

```
TS2322: Type 'T' is not assignable to type 'string | number | readonly string[] | undefined'.
  Type 'unknown' is not assignable to type 'string | number | readonly string[] | undefined'.
    Type 'unknown' is not assignable to type 'readonly string[]'.
      Type 'T' is not assignable to type 'readonly string[]'.
        Type 'unknown' is not assignable to type 'readonly string[]'.
```

I had hoped I could use my `CardinalDirectionType` as it's just strings at least. Even that
won't pass muster. I never did manage to get around it. The internal types are a bit
unforgiving of generics. If I managed to satisfy `value`, `setValue` would start showing errors.
If anyone knows how to do this, I would dearly love to know.
Please [tweet](https://twitter.com/lancegliser) or email me.


Using Typescript generic type variables in ReactFunction components


I've been working on a couple type of code bases in Typescript for our company
that use direct
[`tsconfig.json`](https://www.typescriptlang.org/tsconfig) files instead of inherited
versions. I noticed a off and on but never really let it bug me that the behavior of
`test.ts` / `spec.ts` ([Jest](https://jestjs.io/)),
or `stories.tsx` ([Storybook](https://storybook.js.org/))
was always different than our `src` code.

I recall one time starting to investigate
by trying to change the [`exclude`](https://www.typescriptlang.org/tsconfig#exclude) settings
as I knew we were ignoring those types of files via patterns. Just including those files
back into considering broke our builds though, as we changed the actual build output from
just the content of `src` published to `dist` or `lib`. Instead, with the exclusions removed
the build directories now included their own `src` and `test` folders. No good. With
too little time available to really dig into the issue, I dropped it. After all,
the thing compiled; it was a very minor syntax issue or ignore note in just non-production
files.

Recent work on a starting a [Lerna](https://lerna.js.org/) monorepo that will offer
our clients Typescript SDK's via NPM packages forced me to confront the issue more head on.
That breakdown, of fighting to get NPM, TS, and Jest to all play nice is a much larger battle
still ongoing. I'll post it once I get it all working.

One small tidbit I do want to pull out separately though is the concept of multiple
`tsconfig.json`. Each package in the monorepo required its own as the
[`rootDir`](https://www.typescriptlang.org/tsconfig#rootDir) changed.
This meant either a full copy paste (ick) or using
[`extends`](https://www.typescriptlang.org/tsconfig#extends). Obviously, I choose
to inherit all the possible configurations I could from the root `tsconfig.json`.
In doing this I noticed that the `includes` and `excludes` were wrong of course.

This lead me to the idea of keeping a root `tsconfig.base.json` version with the
`compilerOptions` just as they suggest in the docs for `extends`. Each package then had
its own `tsconfig.build.json` and I just updated the `build` commands to use the
`-p ./tsconfig.build.json` argument where I could use `exclude` for spec, test, and story
files. Great! One last problem though, the IDE has no idea what my options are. There lies
your last `tsconfig.json` file, the one that used to be my default in the project root.
It's nothing more than a `extends` in most cases, but it makes IDE's and allows little tweaks.

As example, here's my current 3 usages for an API project:

## `tsconfig.base.json`

```json
{
  "compilerOptions": {
    "allowJs": true,
    "allowSyntheticDefaultImports": true,
    "declaration": true,
    "declarationDir": "./lib",
    "outDir": "./lib",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "jsx": "react",
    "isolatedModules": false,
    "lib": ["dom", "dom.iterable", "esnext"],
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "strict": true,
    "target": "es5"
  }
}
```

## `tsconfig.json`

```json
{
  "extends": "./tsconfig.base.json",
  "include": [".storybook", "src", "stories"],
  "exclude": ["node_modules"]
}
```

## `tsconfig.build.json`

```json
{
  "extends": "./tsconfig.base.json",
  "include": ["src"],
  "exclude": ["node_modules", "**/*.test.*", "**/*.stories.*"]
}
```


Why you should have 3 tsconfig.json files


Creating pipelines is a funny business. There are several concerns to watch for
that would be standard in version control, but require special work instead.
I'd like to review the idea of a [SemVer'esk](https://semver.org/) major version
control system today. I'll be covering
[Azure DevOps pipelines](https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops),
specifically,
[multi-stage](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/stages?view=azure-devops&tabs=yaml)
[deployment](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/deployment-jobs?view=azure-devops)
[artifact](https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/build-artifacts?view=azure-devops&tabs=yaml)
pipelines with
[templates](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops).
If you need a general primer, I highly recommend their
[Create a multistage pipeline by using Azure Pipelines](https://docs.microsoft.com/en-us/learn/modules/create-multi-stage-pipeline/1-introduction)
module. It will cover the basics of yaml, stages, jobs, tasks, and deployments
that I'll only glance over here.

The example module is targeted at one repository, one pipeline, and a few
deployment targets all known by the pipeline authors. It breaks down once
you start producing generic artifacts without knowing what teams or projects
might consume them. A few use cases we've found so far all relate to
wrapping our own standards around generic open source projects. Thus far,
we've wrapped ArangoDB, Prometheus monitoring stack, Elastic stack, and other
helm charts with variable sets.

## Setting up your publishing pipeline

Generic artifacts created by pipelines do _not_ inherit
[git tags](https://git-scm.com/book/en/v2/Git-Basics-Tagging). They do however allow
you to manually tag the artifact runs using the "more actions triple dot"

The tags work a little differently than standard git usages:

- You can use the same tag multiple times. We'll be using that feature for publishing!
- Any subscribers will pick up the _latest_ pipeline with the specified tag.
  If you tag an old release later, you can cause regressions.
- While you can make full SemVer tags with 3 octets, subscribers can not use them.
  For this reason, we'll use major tags such `1.x`, `2.x`, etc to manage subscribers.

Each time your major tag is added to a release, subscribers will have the option
of automatically trigger a new run.

Here's an example publishing pipeline. I tend to use `azure-pipelines/release.yaml`:

```yaml
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
  batch: true
  branches:
    include:
      - main

variables:
  vmImage: ubuntu-latest

stages:
  - stage: Publish
    displayName: "Publish the base artifact"
    jobs:
      - job: "Publish"
        pool:
          vmImage: $(vmImage)
        displayName: "Prepare job"
        steps:
          - task: CopyFiles@2
            displayName: "Copy helm charts"
            inputs:
              Contents: "helm"
              TargetFolder: $(Build.ArtifactStagingDirectory)
          - task: CopyFiles@2
            displayName: "Copy helm charts"
            inputs:
              Contents: |
                helm/**
                README.md
              TargetFolder: $(Build.ArtifactStagingDirectory)
          - task: PublishBuildArtifacts@1
            inputs:
              PathtoPublish: $(Build.ArtifactStagingDirectory)
              ArtifactName: "drop"
              publishLocation: "Container"
```

## Setting up your subscription pipelines

Your pipeline produces an artifact that can be subscribed to
from other project's pipelines and customized for specific use.
As an example, I'll include parts of our
[ArangoDB helm charts](https://www.arangodb.com/docs/stable/deployment-kubernetes-helm.html).

To do so:

- Create your own new repository in the consuming project.
- Create a subscriber pipeline. I tend to use `azure-pipelines/deployment/pipeline.yml`
- Include a
  [`build resource`](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/resources?view=azure-devops&tabs=schema#define-a-builds-resource)
  dependency on your publishing `build artifact`:

```yaml
resources:
  pipelines:
    - pipeline: _ArangoChart # Used to download the artifact to local files: $(Pipeline.Workspace)/_ArangoChart
      project: Deployment and Hosting # Any project name
      source: arango-chart # The actual pipeline in the project
      # One or more tags.
      # You could use "latest", or any static tag.
      # I would suggest major versions to control breaking changes.
      tags:
        - 2.x
      # Include a trigger to allow automatic updating if desired
      trigger:
        tags:
          - 2.x
```

- Include a deployment stage

```yaml
variables:
  vmImage: ubuntu-latest
  image: arangodb/arangodb:3.7.6
  releaseName: some-release-name

stages:
  - stage: Development
    displayName: "Deploy to the dev environment"
    jobs:
      - deployment: Deploy
        pool:
          vmImage: $(vmImage)
        environment: some-cluster.some-resource
        variables:
          - name: hostName
            value: "dev.arango.example.com"
            readonly: true
        strategy:
          runOnce:
            deploy:
              steps:
                - template: "templates/helm-db-upgrade.yml"
                  parameters:
                    namespace: $(Environment.ResourceName)
                    releaseName: $(releaseName)
                    environment: Development
                    image: $(image)
                    hostname: $(hostName)
                    agentsCount: 2
                    dbserversCount: 2
                    coordinatorsCount: 2
  - stage: Production
    displayName: "Deploy to the production environment"
    jobs:
      - deployment: Deploy
        pool:
          vmImage: $(vmImage)
        environment: some-cluster.some-resource
        variables:
          - name: hostName
            value: "arango.example.com"
            readonly: true
        strategy:
          runOnce:
            deploy:
              steps:
                - template: "templates/helm-db-upgrade.yml"
                  parameters:
                    namespace: $(Environment.ResourceName)
                    releaseName: $(releaseName)
                    image: $(image)
                    hostname: $(hostName)
```

- Include a step in your `azure-pipelines/deployment/templates/helm-db-upgrade.yml`
  to install this chart and customize the values as required.

```yaml
parameters:
  - name: namespace
    type: string
  - name: releaseName
    type: string
  - name: mode
    type: string
    default: Cluster
    values:
      - Cluster
      - ActiveFailover
      - Single
  - name: environment
    type: string
    default: Production
    values:
      - Development
      - Production
  - name: image
    type: string
  - name: imagePullPolicy
    type: string
    default: Always
    values:
      - Always
      - IfNotPresent
  - name: hostName
    type: string
  - name: agentsCount
    type: number
    # Suggested development value: 1
    default: 3 # Based on mode: Cluster defaults
  - name: dbserversCount
    type: number
    # Suggested development value: 2
    default: 3 # Based on mode: Cluster defaults
  - name: coordinatorsCount
    type: number
    # Suggested development value: 1
    default: 3 # Based on mode: Cluster defaults

steps:
  # Ensure the operator is deployed
  - task: HelmDeploy@0
    displayName: "helm upgrade some-arango-operator"
    inputs:
      namespace: ${{ parameters.namespace }} # The K8s namespace
      command: upgrade
      chartType: FilePath
      chartPath: "$(Pipeline.Workspace)/_ArangoChart/drop/helm/operator"
      releaseName: ${{ parameters.releaseName }}-db-operator
      waitForExecution: true
  # Ensure the instance is deployed
  - task: HelmDeploy@0
    displayName: "helm upgrade some-arango-instance"
    inputs:
      namespace: ${{ parameters.namespace }} # The K8s namespace
      command: upgrade
      chartType: FilePath
      chartPath: "$(Pipeline.Workspace)/_ArangoChart/drop/helm/instance"
      releaseName: ${{ parameters.releaseName }}-db-instance
      overrideValues: "spec.mode=${{ parameters.mode }},\
        spec.environment=${{ parameters.environment }},\
        spec.image=${{ parameters.image }},\
        spec.imagePullPolicy=${{ parameters.imagePullPolicy }},\
        spec.agents.count=${{ parameters.agentsCount }},\
        spec.dbserversCount.count=${{ parameters.dbserversCount }},\
        spec.coordinatorsCount.count=${{ parameters.coordinatorsCount }},\
        ingress.hosts[0].host=${{ parameters.hostName }},\
        ingress.tls[0].hosts[0]=${{ parameters.hostName }},\
        ingress.tls[0].secretName=${{ parameters.hostName }}-tls"
      waitForExecution: false
```


Ramblings of a full stack developer

Blog entries

GraphQL file upload mutations

Using layered context functions to share objects for Jest integration testing

Creating dynamic state using state in React

Using Typescript generic type variables in ReactFunction components

Why you should have 3 tsconfig.json files

Creating subscribable Azure DevOps versioned pipeline artifacts