Working with data in Gatsby

23 June, 2019
Time to read: 6 mins
reactgatsby
Last update: 10 July, 2019
Table of contents

In this post, you will see how to work with data in Gatsby from hard coding them, to transformer plugins, and finally to MDX.

Say you want to create an application for a travel agency. The agency organizes 6 tours in different countries over the world. Their homepage should have a section that lists those tours. They also want to show on a different page some extra information about each tour. The first thought that comes to my mind is to create a section on the homepage where each tour will be represented by a card. That section may look like this:

The cards in tours section
The cards in tours section

What’s the best way to implement this requirement?

By the way, I created a repo on GitHub if you want to see the complete code. I'm using styled-system and rebass to style things.

First of all, you have to create the Card component (you can find the code in the link). The card component needs 5 props:

  • A title.
  • A description.
  • A price.
  • A URL that links to the “read more” page.
  • An image.

The first solution is to create in JSX 6 different cards and hard code the data for each tour.

Hard coding the data

To do this, go to the index page, and create 6 different cards. In each card, you pass down the text data as props.

You also want to pass an image. For that, put your images in a folder named src/images/tours, and write a GraphQL query to get them.

The full code for the index page is the following:

src/pages/index.jsx
import React from "react";
import { graphql } from "gatsby";

import { Flex, Box } from "../components/Primitives";
import { Heading1 } from "../components/Variants";
import Layout from "../layouts/Layout";
import SEO from "../components/SEO";
import TourCard from "../components/TourCard";

// eslint-disable-next-line react/prop-types
const IndexPage = ({ data }) => {
  const tours = data.allFile.edges.map(({ node }) => ({
    image: node.childImageSharp.fluid,
    originalName: node.childImageSharp.fluid.originalName,
  }));
  return (
    <Layout>
      <SEO title="Home" keywords={["gatsby", "application", "react"]} />
      <Box height="calc(100vh - 100px)" pt={5}>
        <Heading1 textAlign="center" mb={5}>
          Available Tours
        </Heading1>
        <Flex
          flexWrap="wrap"
          style={{ maxWidth: "1200px", margin: "0 auto" }}
        >
          <TourCard
            title="Spain"
            fluidImage={
              tours.find(({ originalName }) =>
                originalName.match(/spain/i)
              ).image
            }
            price="800$"
            description="1 Week in Spain"
            url="/tours/spain"
          />
          <TourCard
            title="France"
            fluidImage={
              tours.find(({ originalName }) =>
                originalName.match(/France/i)
              ).image
            }
            price="800$"
            description="1 Week in France"
            url="/tours/france"
          />
          <TourCard
            title="Germany"
            fluidImage={
              tours.find(({ originalName }) =>
                originalName.match(/Germany/i)
              ).image
            }
            price="800$"
            description="1 Week in Germany"
            url="/tours/germany"
          />
          <TourCard
            title="Greece"
            fluidImage={
              tours.find(({ originalName }) =>
                originalName.match(/Greece/i)
              ).image
            }
            price="800$"
            description="1 Week in Greece"
            url="/tours/greece"
          />
          <TourCard
            title="Japan"
            fluidImage={
              tours.find(({ originalName }) =>
                originalName.match(/Japan/i)
              ).image
            }
            price="800$"
            description="1 Week in Japan"
            url="/tours/japan"
          />
          <TourCard
            title="USA"
            price="800$"
            description="1 Week in USA"
            url="/tours/usa"
            fluidImage={
              tours.find(({ originalName }) => {
                return originalName.match(/USA/i);
              }).image
            }
          />
        </Flex>
      </Box>
    </Layout>
  );
};

export const query = graphql`
  query TourQuery {
    allFile(filter: { relativePath: { regex: "/tours/" } }) {
      edges {
        node {
          childImageSharp {
            fluid {
              ...GatsbyImageSharpFluid_tracedSVG
              originalName
            }
          }
        }
      }
    }
  }
`;

export default IndexPage;

There is a minor problem with this method. The text data are not linked with the image. That means that you have to write a function to compare the originalName of the image with the current card name, as shown below:

src/pages/index.jsx
<TourCard
  title="Japan"
  fluidImage={
    tours.find(({ originalName }) => originalName.match(/Japan/i)).image
  }
  price="800$"
  description="1 Week in Japan"
  url="/tours/japan"
/>

The other requirement is to create pages with additional information for each tour. For that, you can either create custom pages in src/pages with JSX or create a blog with markdown or MDX. That’s true for every method you’ll see here, and we’ll ignore that requirement for now.

Hard coding the tour data is fine if you don’t have a lot of items. In our case, the items are 6 which is not a small number. If the agency increases the tours in the future, it will become unmanageable.

Before we see the next method, let’s make an improvement on the current one.

Hard coding improvement

If you want to avoid hard coding and writing 6 different cards, you can create a JSON or a JavaScript file that contains the tour data. You can then import that file, and map through it to display the tours.

Create a file with the tour data in src/data/tours.js:

src/data/tours.js
export default [
  {
    title: "Spain",
    price: "800$",
    description: "1 Week in Spain",
    url: "/tours/spain",
  },
  {
    title: "France",
    price: "800$",
    description: "1 Week in France",
    url: "/tours/france",
  },
  {
    title: "Germany",
    price: "800$",
    description: "1 Week in Germany",
    url: "/tours/germany",
  },
  {
    title: "Greece",
    price: "800$",
    description: "1 Week in Greece",
    url: "/tours/greece",
  },
  {
    title: "Japan",
    price: "800$",
    description: "1 Week in Japan",
    url: "/tours/japan",
  },
  {
    title: "USA",
    price: "800$",
    description: "1 Week in USA",
    url: "/tours/usa",
  },
];

Then, import that file in src/pages/index.js, and use it:

src/pages/index.jsx
// skipping
<Flex flexWrap="wrap" style={{ maxWidth: "1200px", margin: "0 auto" }}>
  {tourData.map(({ title, price, description, url }) => {
    const regExp = new RegExp(title, "i");
    return (
      <TourCard
        key={title}
        title={title}
        fluidImage={
          tours.find(({ originalName }) => originalName.match(regExp))
            .image
        }
        price={price}
        description={description}
        url={url}
      />
    );
  })}
</Flex>
// skipping

The code above is better because we avoid duplication. But we still have the problem where the images are not linked with the rest of the data. To solve this, you can use a “transformer” plugin.

gatsby-transformer-json

Another option you have is to take advantage of the Gatsby source plugins. You’ll do almost the same thing you did in the hard coding improvement.

You will place your data in a JSON file, and the gatsby-transformer-json plugin will transform them into “tour” objects. Then, you will write a GraphQL query to get those objects.

If you prefer the YAML format over JSON, you can use the YAML plugin.

You start by installing the plugin:

yarn add gatsby-transformer-json

Then, you configure it in your gatsby-config.js:

gatsby-config.js
const pkg = require("./package");

module.exports = {
  siteMetadata: {
    title: pkg.name,
    description: pkg.description,
    author: pkg.author,
  },
  plugins: [
    "gatsby-plugin-react-helmet",
    "gatsby-transformer-json",
    {
      resolve: "gatsby-source-filesystem",
      options: {
        path: `${__dirname}/src/data`,
      },
    },
    {
      resolve: "gatsby-source-filesystem",
      options: {
        name: "images",
        path: `${__dirname}/src/images`,
      },
    },
    // skipping
  ],
};

The next thing you have to do is to you create a file in src/data/tours.json with the following content. Remove the JavaScript file you added earlier, or transform it into a JSON file:

Notice the highlighted image fields:

src/data/tours.json
[
  {
    "title": "Spain",
    "price": "800$",
    "description": "1 Week in Spain",
    "url": "/tours/spain",
    "image": "images/tours/spain.jpg"
  },
  {
    "title": "France",
    "price": "800$",
    "description": "1 Week in France",
    "url": "/tours/france",
    "image": "images/tours/france.jpg"
  },
  {
    "title": "Germany",
    "price": "800$",
    "description": "1 Week in Germany",
    "url": "/tours/germany",
    "image": "images/tours/germany.jpg"
  },
  {
    "title": "Greece",
    "price": "800$",
    "description": "1 Week in Greece",
    "url": "/tours/greece",
    "image": "images/tours/greece.jpg"
  },
  {
    "title": "Japan",
    "price": "800$",
    "description": "1 Week in Japan",
    "url": "/tours/japan",
    "image": "images/tours/japan.jpg"
  },
  {
    "title": "USA",
    "price": "800$",
    "description": "1 Week in USA",
    "url": "/tours/usa",
    "image": "images/tours/usa.jpg"
  }
]

With that image field, you can now place your tour images in the src/data/images/tours folder. They will be available in GraphQL queries as childImageSharp fields inside the tour objects. You no longer have to search for the images.

The gatsby-transformer-json plugin created for us 2 GraphQL types: The allToursJson and toursJson.

Because we want all the tours, write in your index page a GraphQL query that uses the allToursJson:

export const query = graphql`
  query TourQuery {
    allToursJson {
      edges {
        node {
          description
          price
          title
          url
          image {
            childImageSharp {
              fluid {
                ...GatsbyImageSharpFluid_tracedSVG
              }
            }
          }
        }
      }
    }
  }
`;

With this query in place, your tour data will be available in the data.allToursJson field. You now have to go to the index page, and display the result:

src/pages/index.jsx
// skipping imports

const IndexPage = ({ data }) => {
  const tours = data.allToursJson.edges.map(({ node }) => {
    const { description, price, title, url, image } = node;
    return {
      description,
      price,
      title,
      url,
      image: image.childImageSharp.fluid,
    };
  });
  return (
    <Layout>
      <SEO title="Home" keywords={["gatsby", "application", "react"]} />
      <Box height="calc(100vh - 100px)" pt={5}>
        <Heading1 textAlign="center" mb={5}>
          Available Tours
        </Heading1>
        <Flex
          flexWrap="wrap"
          style={{ maxWidth: "1200px", margin: "0 auto" }}
        >
          {tours.map(({ title, price, description, url, image }) => (
            <TourCard
              key={title}
              title={title}
              fluidImage={image}
              price={price}
              description={description}
              url={url}
            />
          ))}
        </Flex>
      </Box>
    </Layout>
  );
};

To recap, with this method you accomplish 3 things:

  • You separate your data from your UI.
  • You avoid code duplication.
  • You link the images with the data.

But we still have the other requirement. To remind you, the client wants you to create pages with extra information for each tour. We conveniently ignored that requirement, but we’ll have to address it now.

In other words, you’ll have to create something like a blog where the blog post is a tour. A good solution for that requirement is to use the gatsby-mdx plugin.

Using MDX

Instead of MDX, you can use the traditional markdown files. The problem with markdown files is that all the tour “posts” will have the same structure. For example, you can’t add an interactive React component, say an accordion, in the middle of a post.

On the other hand, with MDX you can squeeze in React code if you need to. The idea here is to create an MDX file for each tour where:

  • The file will have frontmatter fields instead of JSON fields to store data for each tour.
  • For each MDX file, you will create a new page with additional information.
  • You will be able to add interactive elements inside the markdown text.

To use MDX in Gatsby you need to install the following packages:

yarn add @mdx-js/mdx @mdx-js/react gatsby-mdx

You can optionally install some remark packages to help with the formatting of the markdown:

yarn add gatsby-remark-images gatsby-remark-smartypants gatsby-remark-copy-linked-files gatsby-remark-external-links

You also have to configure the plugin in your gatsby-config.js:

gatsby-config.js
module.exports = {
  siteMetadata: {
    title: `Travel Agency`,
    description: `Create a travel agency site to see theme's limitations.`,
    author: `Mark`,
  },
  plugins: [
    // skipping
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `tours`,
        path: `${__dirname}/content/tours`,
      },
    },
    {
      resolve: "gatsby-source-filesystem",
      options: {
        name: "destinations",
        path: `${__dirname}/content/destinations`,
      },
    },
    {
      resolve: `gatsby-mdx`,
      options: {
        extensions: [`.mdx`, `.md`],
        defaultLayouts: {
          default: `${__dirname}/src/components/layouts/Layout.jsx`,
        },
        gatsbyRemarkPlugins: [
          {
            resolve: "gatsby-remark-images",
            options: {
              maxWidth: 1920,
              linkImagesToOriginal: false,
            },
          },
          {
            resolve: "gatsby-remark-smartypants",
            options: {
              dashes: "oldschool",
            },
          },
          { resolve: "gatsby-remark-copy-linked-files", options: {} },
          {
            resolve: "gatsby-remark-external-links",
            options: {
              target: "_blank",
              rel: "noopener",
            },
          },
        ],
      },
    },
    // skipping
  ],
};

Now, if you place your MDX files inside the src/pages folder, the plugin will create pages for those files out of the box.

But if you do that, the URL for the tour in Spain will be https://example.com/spain-tour. I want it to be: https://example.com/tours/spain. You may also want to have unpublished tours (drafts), or the client may want to add trips to single destinations instead of only having tours. That’s why I instruct the source filesystem plugin to look inside the content/destinations folder for MDX files in the gatsby-config.js.

So, if you want to place the tour pages in a custom path, you’ll have to configure yourself the page creation in your gatsby-node.js. There are 2 steps here:

  • Generate slugs for the MDX pages (onCreateNode API).
  • Create the pages (createPages API).

I will explain what happens after the following code snippet:

gatsby-node.js
/* eslint-disable no-console */
const path = require("path");
const { createFilePath } = require("gatsby-source-filesystem");

// 1. Create those slugs from file names.
exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions;

  if (node.internal.type === "Mdx") {
    const value = createFilePath({ node, getNode });

    const frontmatterTypes = ["tour", "destination"];
    const frontmatterType = node.frontmatter.type;
    if (frontmatterTypes.includes(frontmatterType)) {
      const parentPath =
        frontmatterType === "tour" ? "tours" : "destinations";
      createNodeField({
        name: "slug",
        node,
        value: `/${parentPath}${value}`,
      });
    }
  }
};

// 2. Create pages
exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions;

  const result = await graphql(
    `
      {
        allMdx(filter: { frontmatter: { published: { eq: true } } }) {
          edges {
            node {
              id
              fields {
                slug
              }
              frontmatter {
                type
              }
            }
          }
        }
      }
    `
  );

  if (result.errors) console.log(result.errors);

  result.data.allMdx.edges.forEach(({ node }) => {
    const tourTemplate = path.resolve(`./src/templates/Tour.jsx`);
    const destinationTemplate = path.resolve(
      `./src/templates/Destination.jsx`
    );
    const template =
      node.frontmatter.type === "tour"
        ? tourTemplate
        : destinationTemplate;
    createPage({
      path: node.fields.slug,
      component: template,
      context: { id: node.id },
    });
  });
};

In the first step, you create the slugs for the MDX pages, and you store them as node fields.

In the second step, you create the pages. You get all the published MDX nodes, and, for each of them, you create a page where you pass:

  • The slug you created earlier as page path. This is the URL that you type in the browser to get the page.
  • The node id (gatsby-mdx created that) as an id field using the context. This is used by a GraphQL query in the template to get the correct page.
  • You specify the appropriate template depending if the MDX file is a tour or a destination.

With this configuration, you can create pages for tours and destinations. But you won’t display the last in the UI, or create MDX files for them. I added the logic just to show what you can do in a gatsby-node.js file.

The next thing you’ll have to do is to create the templates in src/templates/Tour.jsx and src/templates/Destination.jsx (The code is almost the same). I won’t list the code for the templates here, check the GitHub links.

Now, you may want to organize your MDX files in the following way. Each tour will have a folder where inside that folder will be an index.mdx file and the images the tour post needs:

content
|__ tours
|____ france
|______ france.jpg
|______ index.mdx
|____ germany
|______ germany.jpg
|______ index.mdx
|____ greece
|______ greece.jpg
|______ index.mdx
|____ japan
|______ japan.jpg
|______ index.mdx
|____ spain
|______ spain.jpg
|______ index.mdx
|____ usa
|______ usa.jpg
|______ index.mdx
|
|__ destinations

For example, the MDX file for the tour in France may look like this:

---
title: France
description: France tour
type: tour
published: true
date: "2019-06-20T15:15:10.284Z"
image: ./france.jpg
price: 800$
duration: 7 days
destination: Paris
travelers: 45+
---

## Experience France first hand

import { Box } from "../../../src/components/Primitives";

<Box p={4} bg="#1d1d1d" color="white">
  Too bored at this point to add actual content but you get the idea.
</Box>

It’s a tedious task to create all the files yourself, so I suggest to clone the repo, checkout to the mdx branch, and inspect the result.

The last two things you have to do is to change the GraphQL query and the way you map the data in the index page:

src/pages/index.jsx
// skipping imports

const IndexPage = ({ data }) => {
  const tours = data.allMdx.edges.map(({ node }) => {
    const { description, price, title, image } = node.frontmatter;
    return {
      description,
      price,
      title,
      url: node.fields.slug,
      image: image.childImageSharp.fluid,
    };
  });
  return <Layout>{/* skipping */}</Layout>;
};

export const query = graphql`
  query Tours {
    allMdx(
      filter: {
        frontmatter: { type: { eq: "tour" }, published: { eq: true } }
      }
    ) {
      edges {
        node {
          fields {
            slug
          }
          frontmatter {
            title
            price
            description
            image {
              childImageSharp {
                fluid {
                  ...GatsbyImageSharpFluid
                }
              }
            }
          }
        }
      }
    }
  }
`;

export default IndexPage;

You can also check out a similar tutorial on Egghead by Jason Lengstorf.

This was an unexpectedly long post, and I kind of regret starting it (lol). Anyway, before you leave you can see some limitations this solution has in the next section.

Limitations

  • The current solution with the MDX files doesn’t support multiple languages. I’m sure that a travel agency will want their content to be available in a bunch of different languages. To do that, you’ll have to change your gatsby-node.js file. I may add another section in the future (honestly, I won’t) on how to do this. At the very least, I can add a branch in the GitHub repo. For now, you can take a look at an internationalization guide in Gatsby I wrote previously.
  • If the client wants to edit and add new tours or destinations, the MDX files probably won’t cut it. You may have to use a CMS as your data source.

Other things to read

Popular

Previous/Next