Building a UI form

This page is a tutorial on how you can code a generic frontend form which works with all Tray connectors and operations.

It will give you a head start in building a UI form that you will ultimately present for your End Users.

The code used for this tutorial is what drives our Connector Tester app

You can clone the code from the Connector Tester GitHub repo.

It is highly recommended that you install and run the demo app in order to play with the services you plan to use in your application.

This will give you a good understanding of what is involved in terms of the options presented

Tools used

JSON Schema

We use JSON Schema to decribe the data formats for inputs and outputs of our connectors.

In a nutshell, JSON Schema is a declarative language to annotate and validate a JSON document with standard definitions.

If you are interested in reading more about JSON Schema, you can check out their step-by-step guide here.

React + MUI

To build the UI of the app.

Express.js

To build a proxy server that powers UI.e

High level diagram

ui-form-HLD

Step by step guide:

1. Prepare the Input schema for form

As mentioned, every operation on a Tray connector has a predefined input format declared using JSON schema. You can use this input schema directly or modify it to only include the fields you would want to render.

To render the schema, the app uses a popular JSON Schema renderer library called RJSF

info

JSON schema is an industry standard and hence there are several other open source libraries worth exploring, you could also code your own JSON Schema renderer component.

You can also merge schemas of two operations if you are building integrations between two services.

Click here to see the input schema for an integration that pushes contacts from a Google Spreadsheet to Mailchimp:

sheets-to-mailchimp-schema

2. Render the Input schema

Here's a code sample for creating a schemaRenderer component using RJSF's MUI theme. RJSF provides support for multiple CSS libraries including MUI, Bootstrap, Chakra and many more so you can choose the one that fits into your tech stack.

Copy
Copied
import { useState } from "react";
import validator from "@rjsf/validator-ajv8";
import Form from "@rjsf/mui";

export const schemaRenderer = ({ inputSchema }) => {
  const [inputPayload, setInputPayload] = useState({});

  return (
    <Form
      schema={inputSchema} //schema can be passed as a prop from App.js
      validator={validator}
      formData={inputPayload}
      onChange={async (e) => {
        //logic for Updating the schema
      }}
      onSubmit={(e) => {
        //logic for Calling the connector
      }}
    />
  );
};
info

Apart from RJSF, there are several other open source JSON Schema renderer libraries that might be useful for your tech stack:

  1. jsonforms.io - Supports React, Angular and Vue
  2. jsonform - Plain HTML5
  3. angular-schema-form - Angular only
  4. vue-json-schema-form - Vue only

3. Take user auth

For calling a Tray connector on behalf of your user, you would need the user's authId for that service.

Pre-existing Auths

You may have pre-existing auths for that connector owned by the end user. To obtain the user auths, you can use GET user authentications (user-token).

Copy
Copied
async function getAuthentications(bearerToken) {
  const config = {
    headers: {
      Authorization: `Bearer ${bearerToken}`,
    },
  };
  const response = await axios.get(`${API_URL}/authentications`, config);
  return response?.data?.data?.viewer?.authentications?.edges;
}

Note that, This API call will give you all auths owned by the end user. You will have to filter the auths using service name and service version for the connector that you are using.

Here's how you can do it:

Copy
Copied
async function getConnectorAuthentications(service, version, authentications) {
  const filteredList = authentications
    .filter(
      (auth) =>
        auth.node.service.name === service &&
        auth.node.service.version == version
    )
    .map((connectorAuth) => {
      return {
        id: connectorAuth.node.id,
        name: connectorAuth.node.name,
      };
    });
  return filteredList;
}

Service name and version can be obtained from GET connectors

service-name-and-version

Once, you have the exiting auths, You can present a dropdown list to your end user where they can select the auth.

select-end-user-auth

New Auths

If the end user doesn't own an existing auth for the given service, you can present them with a button that will open the auth-only dialog upon click.

For this, you will have to assemble the auth-dialog URL, Refer this page for more details.

The auth only dialog can be whitelabelled with your branding to include your URL.

The auth dialog opens as a popup and sends events to the window from where we opened it. These events are helpful to perform error handling (user closes the auth dialog without completing it etc.) on a failure event and to capture the authId from a sucecss event. You can find the list of events here.

Here's a code sample on how this can be implemented:

Copy
Copied
export const openAuthWindow = (url) => {
  // Must open window from user interaction code otherwise it is likely
  // to be blocked by a popup blocker:
  const authWindow = window.open(
    undefined,
    "_blank",
    "width=500,height=500,scrollbars=no"
  );
  const onmessage = (e) => {
    console.log("message", e.data.type, e.data);
    if (e.data.type === "tray.authPopup.error") {
      // Handle popup error message
      alert(`Error: ${e.data.error}`);
      authWindow.close();
    }
    if (
      e.data.type === "tray.authpopup.close" ||
      e.data.type === "tray.authpopup.finish"
    ) {
      authWindow.close();
    }
  };
  window.addEventListener("message", onmessage);

  // Check if popup window has been closed
  const CHECK_TIMEOUT = 1000;
  const checkClosedWindow = () => {
    if (authWindow.closed) {
      window.removeEventListener("message", onmessage);
    } else {
      setTimeout(checkClosedWindow, CHECK_TIMEOUT);
    }
  };

  checkClosedWindow();
  authWindow.location = url;
};

const authDialogURL = `https://${AUTH_DIALOG_URL}/external/auth/create/${PARTNER_NAME}?code=${json.data?.generateAuthorizationCode?.authorizationCode}&serviceId=${serviceId.current}&serviceEnvironmentId=${selectedServiceEnvironment.id}&scopes[]=${scopes}`;

openAuthWindow(authDialogURL);

4. Updating input schema with independent DDLs

DDLs are dynamic dropdown lists which means they are essentially dropdown lists (<select> tag in HTML) bu they are dynamic as the options in the list would change if using a different auth or unser input from a different field.

Every DDL operation contains a lookup key with the operation name that's powering it and the inputs required for that operation.

Independent DDLs (Dynamic dropdown lists) in Tray refer to the DDL operations that are dependent only on the user auth but don't change on user inputs.

For example, channel field in Slack is a dropdwon list. The lookup opeation - list_users_and_conversations_ddl does not require any user inputs. This operation requires user auth to list the slack channels from their workspace.

An independent DDL's input field will be an empty object as shown below:

Copy
Copied
...
...
"channel": {
  "type": "string",
  "description": "The user or channel the message is being sent to.",
  "lookup": {
    "operation": "list_users_and_conversations_ddl",
    "input": {}   //blank input object
  },
  "title": "Channel"
}
...
...

As soon as the user selects an auth on your form, you should populate all independent DDLs with their enum list values.

Here's how you may do it in your codebase:

Copy
Copied
async function populateDDLSchema(inputSchema) {
  for (let key in inputSchema.properties) {
    // JSONata expression to check for deeply nested key
    const expression = jsonata("**.lookup");
    // check if key contains a `lookup` key within it
    const result = await expression.evaluate(inputSchema.properties[key]);
    // check if the lookup is an independent DDL (ie. input is an empty object)
    if (typeof result === "object" && JSON.stringify(result.input) === "{}") {
      const body = {
        operation: result.operation,
        authId: selectedAuthentication.id, // selectedAuthentication is a state value of the app
        input: {},
        returnOutputSchema: false,
      };

      // call the proxy server. selectedConnectorName and selectedConnectorVersion are state values of the app
      const response = await axios.post(
        `${API_URL}/connectors/${selectedConnectorName}/versions/${selectedConnectorVersion}/call`,
        body,
        {
          headers: {
            Authorization: `Bearer ${token}`, // token is a state value of the app
            "Content-Type": "application/json",
          },
        }
      );
      /*
      response of the lookup operation is an array of objects in the following format:
      [
        {
          value: "",
          text: ""
        },
        {
          value: "",
          text: ""
        }
        ...
      ]
      */
      // prepare an array of option values from the result of lookup.
      const enums = await jsonata(`[output.result.value]`).evaluate(
        response?.data
      );
      // prepare an array of option labels from the result of lookup.
      const enumNames = await jsonata(`[output.result.text]`).evaluate(
        response?.data
      );
      if (enums.length > 0) {
        /* The JSONata expression below transforms the inputSchema 
        object by adding the enum values and labels at the key level */
        const newInputSchema = await jsonata(`$~>|**|{
          "enum":lookup.operation=$operation?$enumValues,
          "enumNames":lookup.operation=$operation?$enumLabels
          }|`).evaluate(JSON.parse(JSON.stringify(inputSchema)), {
          operation: result.operation,
          enumValues: enums,
          enumLabels: enumNames,
        });
        setInputSchema(() => newInputSchema);
      }
    }
  }
}

5. Updating input schema with dependedent DDLs

A dependent DDL operation is a field whose lookup key has a non-empty input field.

For example, list_worksheets_ddl_return_name is a dependent DDL in the Sheets connector as it's dependent on spreadsheet_id.

Copy
Copied
...
...
"worksheet_name": {
  "type": "string",
  "description": "The name of the sheet inside the spreadsheet.",
  "lookup": {
      "operation": "list_worksheets_ddl_return_name",
      "input": {
        "spreadsheet_id": "{{spreadsheet_id}}"  //input object is non-empty
      }
  },
  "title": "Worksheet name"
}
...
...

Once the user starts filling the form, you will have to write a function to update schema whenever you have all the inputs required for a dependent DDL.

Here's how you may do it in your codebase:

Copy
Copied
// JSONata expression to look for dependent DDL operations. It's looking for all fields which contain a `lookup` and the input key within the lookup is a non-empty object.
const depedentDDLs = await jsonata(`[**.lookup[input!={}]]`).evaluate(
  inputSchema
);
// dependentDDLOperations is a ref value created using useRef hook
dependentDDLOperations.current = depedentDDLs;

// this function is called onChange of the <Form> component (Schema Renderer)
async function checkAndPopulateDependentDDL(formData) {
  for (let i = 0; i < dependentDDLOperations.current?.length; i++) {
    const ddlOperation = dependentDDLOperations.current[i];
    const inputs = Object.keys(ddlOperation.input);

    /* check if all fields required in the input object 
    of the DDL operation are present in the formData */
    if (
      inputs.every(
        (key) =>
          formData[
            ddlOperation.input[key].slice(
              ddlOperation.input[key].lastIndexOf("{") + 1,
              ddlOperation.input[key].indexOf("}")
            )
          ]
      )
    ) {
      // prepare the input object to be sent as the body of the DDL operation API call
      const bodyInputObject = {};
      inputs.forEach(
        (input) =>
          (bodyInputObject[input] =
            formData[
              ddlOperation.input[input].slice(
                ddlOperation.input[input].lastIndexOf("{") + 1,
                ddlOperation.input[input].indexOf("}")
              )
            ])
      );

      const body = {
        operation: ddlOperation.operation,
        authId: selectedAuthentication.id, // selectedAuthentication is a state value of the app
        input: bodyInputObject,
        returnOutputSchema: false,
      };

      // call the proxy server. selectedConnectorName and selectedConnectorVersion are state values of the app
      const response = await axios.post(
        `${API_URL}/connectors/${selectedConnectorName}/versions/${selectedConnectorVersion}/call`,
        body,
        {
          headers: {
            Authorization: `Bearer ${token}`,
            "Content-Type": "application/json",
          },
        }
      );
      // prepare an array of option values from the result of lookup.
      const enums = await jsonata(`[output.result.value]`).evaluate(
        response?.data
      );
      // prepare an array of option labels from the result of lookup.
      const enumNames = await jsonata(`[output.result.text]`).evaluate(
        response?.data
      );
      if (enums.length > 0) {
        /* The JSONata expression below transforms the inputSchema 
        object by adding the enum values and labels at the key level */
        const newInputSchema = await jsonata(`$~>|**|{
          "enum":lookup.operation=$operation?$enumValues,
          "enumNames":lookup.operation=$operation?$enumLabels
        }|`).evaluate(inputSchemaRef.current, {
          operation: ddlOperation.operation,
          enumValues: enums,
          enumLabels: enumNames,
        });
        inputSchemaRef.current = newInputSchema;
        dependentDDLOperations.current.splice(i, 1);
      }
    }
  }
}

6. Handling form submit

You can code the onSubmit event on the form to perform the Call connector operation using the form data.

With RJSF, the formData can be easily accessed from the event object as it's present as a separate key.

Copy
Copied
onSubmit={(e) => {
  callConnector(e.formData);
}}

Your business logic would also go in here if you have to call multiple operations and transform the data in between steps.

Here's an example for the onSubmit event of an integration to migrate contacts from Google Sheets to Mailchimp.

Copy
Copied
onSubmit={async (e) => {
  //destructuting form data
  const {
    spreadsheet_id,
    worksheet_name,
    list_id,
    mapping,
    mailchimp_subscription_status,
  } = e.formData;

  //preparing the mapping object
  const mappingObject = await jsonata(`
  $merge($map($, function($v, $i){
    {
      $v.sheet_fields : $v.mailchimp_fields
    }
  }))
  `).evaluate(mapping);

  // preparing payload for `get_total_active_rows` operation on Sheets connector
  const getActiveRowsPayload = {
    operation: "get_total_active_rows",
    authId: selectedSheetsAuthentication, //selectedSheetsAuthentication is a state value
    input: {
      spreadsheet_id: spreadsheet_id,
      worksheet_name: worksheet_name,
    },
    returnOutputSchema: false,
  };

  // calling the `get_total_active_rows` operation on Sheets connector
  const activeRowsResponse = await callConnector(
    'sheets',
    '8.1',
    getActiveRowsPayload
  );
  const numOfRows = activeRowsResponse.output.rows;

  // preparing payload for `get_rows` operation in Sheets
  const getRowsPayload = {
    operation: "get_rows",
    authId: selectedSheetsAuthentication,  //selectedSheetsAuthentication is a state value
    input: {
      spreadsheet_id: spreadsheet_id,
      worksheet_name: worksheet_name,
      number_of_rows: numOfRows,
      format_response: true,
    },
    returnOutputSchema: false,
  };

  // calling the `get_rows` operation on Sheets connector
  const sheetRows = await callConnector(
    'sheets',
    '8.1',
    getRowsPayload
  );

  const transformedRowsResponse = await jsonata("$.result").evaluate(
    sheetRows.output.results
  );

  const sheetsFields = Object.keys(transformedRowsResponse[0]);

  // the function transforms the data as per the acceptable input schema of Mailchimp's batch subscribe operation
  const membersArray = preparePayloadForMailchimpBatchSubscribe(
    transformedRowsResponse,
    mappingObject,
    sheetsFields,
    mailchimp_subscription_status
  );

  const mailchimpBatchSubcribePayload = {
    operation: "raw_http_request",
    authId: selectedMailchimpAuthentication,
    input: {
      method: "POST",
      include_raw_body: false,
      parse_response: "true",
      url: {
        endpoint: `/lists/${list_id}`,
      },
      query_parameters: [
        {
          key: "skip_duplicate_check",
          value: "true",
        },
      ],
      body: {
        raw: {
          members: membersArray,
        },
      },
    },
    returnOutputSchema: false,
  };

  const resultOfSubmit = await callConnector(
    mailchimp.connectorName,
    mailchimp.connectorVersion,
    mailchimpBatchSubcribePayload
  );
}}