Karrot Documentation

This is the very early stage of some product/developer documentation for Karrot.

Purposes would/could be to document:

  • useful resources/links
  • dev FAQs
  • feature descriptions
  • high level code design concepts
  • our conventions (e.g. enriching vuex getters)

Contents

FAQ

How does this relate to foodsharing.de?

This is new piece of software, targeted at foodsaving groups from around the globe. The page was designed with the experiences of foodsharing.de in mind.

The development teams actually overlap, so communication is very close.

Also, see some answers about this topic.

Karrot Features

This is in detail description of all the features in Karrot. For each feature section it should explain the context, and the philosophy, and the details. Perhaps it's a bit like a spec? But right now an attempt by me to understand the features and have them in one place, so I can think more clearly!

See the entries in the menu on the left for more details.

Groups

Groups are at the heart of Karrot. Each group is independent from the rest and will usually represent an in-person community of people. They normally have their own organisational name and structure.

They are not part of the karrot organisation, they just use karrot. Ideally one of more of the community members participate in our community forum, or via chat.karrot.world/channel/karrot-dev/GitHub/etc.

  • an overview of all the groups on the instance
  • filterable by
    • search (by group name)
    • can hide inactive groups
    • seperates groups you are part of from those you are not (if logged in)
  • world map showing all groups

Group Preview

  • shows details for a single group
  • displays the public description of the group
  • options to join group if not already a member

Create a Group

You can create a group with the follow fields:

field
title
public description
internal description
welcome message
address text
address lat/lon
questions for applicants
timezone

Edit Group

  • same fields as for create group
  • but additionally a group image can be added, which will be displayed in the top left and on the group preview overview

Group Wall

  • see chat/wall section below
  • shows activities you have signed up to at the top
  • shows activities with available slots from your favourite places, default collapsed

Group History

  • an audit of activities by users in this group

Full Page Map

  • can select:
    • places on/off
    • users on/off
    • groups on/off
  • export all selected things to GPX file

Member List

  • sorted by join date or alphabetically
  • by default, inactive users are not shown, but you can expand a section at the bottom to show them

Notifications

Weekly Summary

  • sent out at 8am local time every Sunday
  • provides an overview of all the activites in the group in that week
  • idea borrowed from the slack emails of a similar nature

Group Applications

Karrot groups are closed (with the exception of the "Playground" group which is open to everyone), so you need to apply to join. Group Applications manage this process.

  • for all "closed" groups (everything but the Playground and some older inactive groups) users have to apply to join
  • the user applying is shown the "questions for applicants" (group setting)
  • members of the group can see the list of applications
  • a chat is created with the applicant and the rest of the group (or just "editors"?)
  • members of the group (or just editors?) can accept or decline the application (on the overview page)
  • past applications are hidden on the list by default, but can be expanded

Notifications

typenotificationto whocondition
emailnew application createdgroup membersmember has "new application" notification type enabled
emailapplication accepteduser who applied
emailapplication declineduser who applied

Places

  • within a group

Create a Place

fields:

field
name
status (just created, negotiating, co-operating, don't want to co-operate)
description
address text
address lat/lon
how many weeks in advance should activities be scheduled

Edit Place

  • same fields as above

Place Wall

  • see wall/chat section

Activity Feedback

  • shows feedback about activities for that place

Place History

  • like group history, but just for that place

Manage Activities

  • see activities section

Mark as Favourite

  • can mark certain places as favourite
  • will show up at top of place list in sidebar with a star
  • will subscribe you to place wall
  • shows you available activities from favourite places at top of group wall

Show Directions

  • opens openstreetmap (or something else on mobile) to provide directions to the place

Notifications

  • ?

Activities

  • within a group and a place

Create/Edit One-off Activity

fields:

field
date
time
optional end time
max slots (how many people can participate)
extra information

Create/Edit Recurring Activity

fields:

field
frequency (weekly, or custom rrule)
if weekly, time
if weekly, optional end time
if weekly, which days of the week
if custom, start date
if custom, start time
if custom, custom rule https://www.kanzaki.com/docs/ical/rrule.html
max slots
extra information
  • can edit individual activities in the series
    • slots
    • additional information
    • can disable

Activity List

  • can see who has signed up
  • join/leave
  • chat in sidebar
  • filter by
    • all
    • with free slots
    • only empty

Activity Feedback

  • after a activity has finished you can provide feedback
  • has optional weight and a comment

Notifications

  • the night before, if not filled already
  • reminders for your own ones

Issues

  • a place to discuss and resolve issues in the group
  • can be created via the button on a members profiles if enough trust has been given
  • a multi page wizard before the issue is created to ensure the user understands the process
  • has a "target person" that the issue is about
  • a chat is created with all members (or just editors?) and the person
  • after some time of discussion a score voting process occurs
  • the proposals to vote on are fixed by the software
  • voting can result in
    • leaving the member in the group
    • removing member from group
    • repeat the process (or add more time?)

Offers

  • within a group
  • for offering goods or services to other members of the group

Create Offer

fields:

field
name
description
1 or more photos

List Offers

  • can filter by available, or your own archived offers

Offer Detail

  • chat
  • archive offer
  • edit offer

Notifications

  • "New offers" notification setting to receive emails when new offers are created, defaults to off (I think)

Trust

  • exists between two users, per group
  • some thresholds for getting "editor" role

Account

Create Account

  • fields
    • name
    • email
    • password
  • leads to unverified account

Profile

  • in the context of a group (so you can only see profiles if you have common groups)
  • trust
  • photo
  • email
  • location
  • history
  • link to edit

User Settings

  • profile photo
  • profile fields
    • name
    • description
    • phone number
    • address text
    • address lat/lon
  • language selector
  • account info
    • change email address
    • change password
    • delete account
  • notification settings
    • per group
    • 5 different notification types
    • unsubscribe from all (probably means all the chats/etc inside the group too)
  • push notifications
    • current browser push notifications toggle

General

  • groups, places, users

Messages

  • wall posts, private chats, wall replies, etc...
  • conversations/replies switch

Notifications

  • various activities (a bit similar to history)

About

  • about karrot, info about team, and links to other places

Community Forum Discussions

  • show discussions from the community forum

Chats and Walls

  • built on reusable conversations feature
  • write messages
  • emoji reactions

Walls

  • on groups and places
  • newest message on top
  • threads are like chats on a wall message

Chats

  • newest message on bottom
  • multiple types:
    • application chat
    • private chat
    • activity chat
    • issue chat

Frontend structure

Modules and directories

The Karrot frontend is composed of modules. Each module should follow this directory structure:

src/
  <module-name>/
    - routes.js                 # routes for this app
    - assets/                   # mostly for images used in this module
      - apple.png
    - api/                      # XHR communication, to backend and other services
      - activity.js
      - activity.spec.js          # unit test
      - ...
    - datastore/                # vuex namespaced modules and plugins
      - activity.js
      - activity.spec.js          # unit test 
      - ...
    - components/               # reusable components (atoms, molecules, organism)
      - ActivityUser.vue
      - ActivityUser.spec.js      # unit test
      - Activities.story.js        # storybook story
      - ...
    - pages/                    # page templates and instances
      - ActivitiesManage.vue       # page connected with mapGetters and mapActions
      - ActivitiesManageUI.vue     # or with vuex-connect
      - ...

The modules base and utils stand out as they don't focus on one area. base contains all bits that could be considered the core of Karrot, while utils are helpers that can be reused in other modules. They should be kept as small as possible, so consider creating a new module before adding to them.

Forms

Forms and validation is quite a complex topic! This just covers a few topics for now.

Our approach for forms is to clone an editing version of the data for use in the form. This allows us to:

  • know if the value of the data has changed (diff(source, edit))
  • reset the form (edit = source)
  • emit the diff of changes in a @save event (emit(diff(source, edit))) (works nicely with http PATCH)
  • handle underlying changes in the source value as we wish

In many places we enrich the data with additional fields, but we only want to send unenriched fields to the API. Luckily, the diff of changes provides us exactly that.

editMixin and statusMixin

This logic is encapsulated in editMixin and statusMixin, so you get these features with just:

export default {
  mixins: [editMixin, statusMixin],
}

They require additional props to the component:

  • editMixin requires value: the object you are editing, which must have an __enriched field
  • statusMixin requires status from the meta module

statusMixin provides helpers to quickly access the validation error status, e.g. by field name:

<div v-if="hasError('password')">
  {{ firstError('password') }}
</div>

Request status and server validation errors

Request status and server validation errors are handled using the meta module. We store meta for each combination of action and id (optional), the underlying state looks like this:

{
  byAction: {
    create: {
      pending: false,
      validationErrors: {}
    }
  },
  byId: {
    323: {
      save: {
        pending: false,
        validationErrors: {
          name: ['must be unique']
        }
      }
    }
  }
}

There is only 1 meta getter which you use like this:

// for an action without an id
getters['meta/status']('create')

// for an action with an id
getters['meta/status']('save', id)

It will always return a value, by default it will be:

{
  pending: false,
  validationErrors: {}
  hasValidationErrors: false,
  firstValidationError: undefined, // contains first validation error, if any
  firstNonFieldError: undefined, // contains `nonFieldErrors` or permission errors, if any
}

There is one action, clear, which has the same signature:

// for an action without an id
dispatch('meta/clear', ['create'])

// for an action with an id
dispatch('meta/clear', ['save', id])

Using meta in Vuex modules

Overview

import { createMetaModule, withMeta } from '@/store/helpers'

export const modules = { meta: createMetaModule() }

export const actions = {

  // This wraps the actions so we catch pending status and validation errors
  ...withMeta({

    // If `entry` has an `id` field, we use that
    async save ({ commit }, entry) {
     // ... do stuff ...
    },

    // The argument is a number, we use that as the `id`
    async join ({ commit }, id) {
     // ... do stuff ...
    },

    // No argument, so it is stored by action name
    async list ({ commit }) {
      // ... do stuff ...
    },

  })
}

Getters

When you use it in a Vuex module there are two places to put it:

  • in the enrich method, you can add meta by id for any interesting actions, your enrich method might like like this:
import { metaStatusesWithId } from '@/store/helpers'

export const getters = {
  // ... other getters ...
  enrich: (state, getters) => entry => {
    return entry && {
      ...entry,
      // this will add `saveStatus`, `joinStatus`, and `leaveStatus`
      ...metaStatusesWithId(getters, ['save', 'join', 'leave'], entry.id),
      // ... other enriched fields ...
    }
  }
}
  • as it's own getter for actions without an id, which might look like this:
import { metaStatuses } from '@/store/helpers'

export const getters = {
  // ... other getters ...
  // this will add a `createStatus` getter
  ...metaStatuses(['create']),
}

The convention is to name the fields <actionName>Status, e.g. the status about a save action would be available as saveStatus or the join action as joinStatus

Checklist for creating new forms

  • Does the form edit an existing object? -> use editMixin
  • Does the form show server-side validation errors? -> statusMixin?
    • uses hasAnyError and anyFirstError to show any server-side validation error, server error or network error?
    • uses hasError(field) & firstError(field) methods to check for field-specific server-side validation errors?
    • uses hasNonFieldError & firstNonFieldError to show errors unrelated to fields?
    • sets ::loading="isPending" on submit button?
    • uses pending status for non-submit actions? (e.g. destroy)
  • Does the form use vuelidate ($v) to check for validation errors client-side?

Pages and Routes

Pages are the views of the app, each of them is defined in a route object. It carries some important data:

{
  name: 'groupPreview',
  path: '/groupPreview/:groupPreviewId',
  meta: {
    // custom properties like data loading and breadcrumb definitions
  },
  components: {
    default: GroupPreview,
  },
},

Loading data on page load

Quite often, you will find that a page needs data from the API to show something meaningful. We use a declarative approach to do this. By specifying a vuex action beforeEnter properties on the route meta object, the action will run before the page loads.

The action gets the route params object passed as first argument. In the example below, you can use the groupPreviewId in the action.

{
  name: 'groupPreview',
  path: '/groupPreview/:groupPreviewId',
  meta: {
    beforeEnter: 'groups/selectPreview',
    afterLeave: 'groups/clearGroupPreview',
  },
  components: {
    default: GroupPreview,
  },
},

The beforeEnter action gets awaited, so it's guaranteed to complete before the page loads. Still, better make everything reactive anyways.

If you have nested routes, the actions of the parent will run before the child ones. To pass data from parent to child (e.g. selected group), you can use the vuex state. All actions get the complete route params object.

Show a route error page

If the beforeEnter action determines that the route can't be accesses (e.g. because of invalid IDs in the URL), it can throw a special routeError to show an error page.

You can pass a message to the user along with the error:

if (hasError) {
  throw createRouteError({ translation: 'NOT_FOUND.EXPLANATION' })
}

Clean up after yourself when leaving a page

Similar to beforeEnter, you specify an afterLeave property on the meta object. That makes it possible to clear form errors that should not persist page reloads.

{
  name: 'groupPreview',
  path: '/groupPreview/:groupPreviewId',
  meta: {
    afterLeave: 'groups/clearGroupPreview',
  },
  components: {
    default: GroupPreview,
  },
},

Alerts

We distinguish between banners and toasts.

Banners

Banners are alerts that are potentially shown for a long period of time. For instance, a banner could remind the user to sign an agreement which does not have a deadline.

Banners are shown at the top of the screen, right below the navigation bar.

To create a new banner, first add a new method to the BannerUI Vue component:

newBanner (context) {
  return {
    color: 'some_color',
    icon: 'some_icon',
    message: 'SOME.MESSAGE',
    actions: [
      {
         label: this.$t('SOME.ACTION'),
         handler: () => {
           doSomething()
         },
      },
    ],
  }
}

Then add it to the list of banners in the banners vuex store:

if (newBannerShouldBeShown) {
  banners.push({
    type: 'newBanner',
    context: someContext
  })
}

Note that type has to equal the name of the method created in the first step, and someContext will be passed to it as a parameter. The flag newBannerShouldBeShown controls when the banner should be displayed.

Toasts

Toasts are alerts that are shown only for a very short period of time or that require immediate interaction. Typical examples are simple sucess or error alerts.

A toast is displayed at the bottom of the screen. If it is not closed manually (by clicking the default close button), it disappears automatically after a couple of seconds.

To show a toast, simply trigger the toasts/show action:

dispatch('toasts/show', {
  message: 'SOME.MESSAGE',
  messageParams: { param: value },
  config: { type: 'someType' },
}, { root: true })

Both messageParamsand config are optional.

Internationalization (i18n)

Locales

Karrot is available in different languages. So, instead of adding English text directly to a component's source file, we add a key instead. This key points to a text in each of the locale files.

All locale files can be found under src/locales/. For example the Spanish locale file is located at src/locales/locale-es.json.

Instead of

<ChangePhoto
  ...
  :label="Group logo"
  :hint="Click to upload your group logo!"
  ...
/>

we write

<ChangePhoto
  ...
  :label="$t('GROUP.LOGO')"
  :hint="$t('GROUP.SET_LOGO')"
  ...
/>

$t(...) is a function provided by vue-i18n which does all the lookup magic.

Adding new text to locales

  • Add a new key-value pair to src/locales/locale-en.json
  • As a convention keys are CAPITALIZED.
  • Keys can be nested for better structure.
  • Do not add the key to the other locale-*.json files. They will be added automatically by the CI tool after merge to master and translated by transifex.

Adding new locales

TODO hey stranger, do you know how to add new locales? Improve this documentation by answering this question.

karrot Backend Introduction

  • Website: https://karrot.world
  • Predecessor: https://foodsharing.de
  • Repository on GitHub: https://github.com/yunity/karrot-backend

This is a beginner guide to karrot-backend by @id-gue and @mddemarie written for people who want to contribute to karrot, but aren't (yet) experienced Python/Django devs. Welcome and have fun!

Repository Structure

There are two separated Repos for Frontend and Backend.

Karrot-Frontend in JavaScript https://github.com/yunity/karrot-frontend

You don't need to do the setup for the frontend, but it might be useful to try out your backend through the frontend.

Karrot-Backend in Python Django REST https://github.com/yunity/karrot-backend

  • Python – object-oriented programming language
  • Django – Python framework for backend development Tutorial: https://docs.djangoproject.com/en/1.11/intro/tutorial01
  • Django REST Framework on top of django for building Web APIs Tutorial: http://www.django-rest-framework.org/tutorial/1-serialization

Both repositories are not directly connected – the data exchange works via an API.

01 Setup

We use Docker for the setup. How to build a Docker container is described in the README.md in the karrot-backend repository.

We would suggest to use 3 tabs in the shell:

  1. Tab for Communicating with git / GitHub (doing that inside the docker container might raise errors)

  2. Tab with Docker running for run manage.py commands

    Find out the name of your Docker Container: docker ps (examples: young_curie or amazing_lovelace)

    Run Docker with: docker exec -it <container_name> bash (After starting Docker your lines in the shell start with: (env))

    Running tests:

    python manage.py test (Please run the tests after your setup and every time you make a change in code.)

    After changing a model you have to migrate them:

    python manage.py makemigrations python manage.py migrate

    Leave Docker: exit

  3. Tab with Docker running to check what your server is doing

    Show the last 12 lines of the server output: docker logs -f <container_name> --tail "12"

    Note: The first line shown is an email address. Store it – we will need it for Swagger.

02 Project Architecture

Relationships in Backend

First of all, you have to have a Group Model, allowing to create objects like "Foodsavers Berlin". One Group usually has many Stores, like "Bakery Smith". Each store can define events where foodsavers can come by and save food. These events are called PickupDate (one time event) or PickupDateSeries (repetitive event).

core elements of foodsaving backend

As logged-in user, you can create and join a Group, what makes you a member. Afterwards, you can join or create a PickupDate event which takes place in the future, what makes you a collector.

Further actions are for example:

  • for member in Group: create/modify/join/leave
  • for member in Store: create/update/delete
  • for collector in PickupDate/PickupDateSeries: create/join/update/delete

Collectors have also an option after food pickup to leave feedback.

Foodsaving Apps

At the moment (September 2017) there are 15 Apps (= folders) in foodsaving. Not all of them are in use or critical for karrot.world since the project is under development and the dev team tries different approaches.

Important apps are for example:

  • groups (see above)
  • users (user data and user profile, reset user password, change password etc.)
  • userauth (login and logout)
  • base (most models in the code inherit from the models created there)
  • tests (the test coverage is very high - some of the tests are in the test app – others in the other apps)
  • stores and history might need a bit more explanation:

Stores

In models.py in stores, you can find classes for Stores and Feedback, as well as PickupDate and PickupDateSeries. The last two refer to pickup-date and pickup-series in Swagger (see chapter "Server and Swagger") and contain appropriate data fields. PickupDateManager with the method process_finished_pickup_dates is an interesting class because it processes old pickups and moves them into history (even empty ones) - as a result you find PICKUP_MISSED or PICKUP_DONE in the database.

History

In history you find any action regarding stores, groups or pickup-dates/pickup-series from the past. As a result, you find here different HistoryTypus (just “typus” in database), e.g. PICKUP_JOIN and additional data about that action. This helps to keep a track of all actions.

03 Stores app in detail

We want to dig a bit deeper into the app Stores (a) to give you an example of how the foodsaving apps work and (b) because there is too much functionality inside that you might like to know. If you haven't already opened the code in your editor: do it now! Open the stores app and have a look at the files:

  1. models.py Here you define which database tables you want to have and what the fields/columns should store in the database. One model (or class) defines one database table. Let's have a look at the model Feedback which creates four database fields (and two fields for the id and a time stamp, but these are created automatically here). The following line creates a field with the name comment.

    comment = models.CharField(max_length=settings.DESCRIPTION_MAX_LENGTH, blank=True)

    The type CharField says that comment will be stored as string in database. The maximum string length is given as DESCRIPTION_MAX_LENGTH in the file settings.py. The entry can be saved even if the comment field is blank.

  2. serializers.py The models we created in models.py are python objects. But these are not very useful in order to access the API – so we convert them to JSON objects with serializers. Our Feedback model has a FeedbackSerialzer which inherits many functions from ModelSerializers. But there are also new functions like validate_about. (user is a member of group, that member joined the pickup and the pickup is in the future). This validator checks if a user is allowed to give feedback about a certain pickup. (Validation within a Serializer might sound strange, but it's common in the REST framework. See Validators in the documentation)

  3. permissions.py Another possibility to check if something is allowed are permissions. They are used in api.py. Here is for example the permission IsNotFull that permits a member to join the pickup event only if it is not full.

  4. api.py The api defines how the data stored in the database can be accessed via API. The used HTTP methods (like GET, POST or PATCH) are described in chapter 03 Server and Swagger.

    Instead of normal Views we use whole ViewSets which allow to combine the logic for a set of related views. Have a look on the class FeedbackViewSet. You will notice that most HTTP methods (like GET) are not defined there but in an imported mixin. Each mixin contains whole logic for creating a single HTTP request. The ViewSets are connected with urls.py and defined there in form of a url.

  5. factories.py In a Factory you can create sample data used in the tests.

  6. A folder with tests The test coverage of the project is very good and Circle CI will answer in angry red if you try to push untested or non-functioning code.

    Have a look on the class FeedbackTest in test_feedback_api.py. First we create all data we need in the setUpClass we are going to use in our tests. Then we test step by step if the expected result is assertEqual to the actual result. (The chapter '01 Setup' explains how to run the tests in the shell.)

  7. A folder with migrations: You don't have to care about them a lot here. They are generated automatically when you run python manage.py makemigrations in the shell with Docker active.

Please also have a look on the used urls in config/urls.py and on the archive functions in foodsaving/history.

04 Server and Swagger

Why do you need the server output in the shell?

On one hand, this way you will notice when an error occurred (or worse, when the server crashed). On the other hand, you can observe the communication with the server while you interact with it.

Furthermore, you see an automatically generated mail address when you start running your docker container (see chapter Setup). Use this and the password 123 to login to Swagger in your browser:

http://127.0.0.1:8000/docs/

Why Swagger?

Swagger shows you the API endpoints that are defined in the api.py files in the apps groups, stores etc. One of the API endpoint is pickup-dates.

You can use HTTP methods like:

  • GET: query data from database
  • POST: submits new entry into database
  • PATCH: modifies one entry in database based on given id
  • DELETE: deletes one entry from database based on gived id

You can also add additional functionalities to your API endpoint like:

  • GET /api/..../{id}/: displays one entry from database based on given (e.g. pickup-date) id
  • POST /api/..../{join}/: the user/member joins the group/store/pickup
  • POST /api/..../{leave}/: the user/member can leave the group/store/pickup
  • any other functionality added to GET, POST, PATCH or DELETE

The Database is automatically populated with sample data if you use Docker. But there are missing connections between: being user -> being member -> being collector -> pick up the food. You can create these connections in Swagger for testing purposes.

TIP: If you want, you can populate the database writing some querysets in the Django Shell and then look it up in Swagger. Or you can open PostgreSQL and populate the database there.

Response-request Cycle

Whenever you paste the url http://127.0.0.1:8000/docs/ into your browser and hit Enter, you send a request to your local server sitting on your computer (live web sites have their own host server and domain). It depends if you want to GET data or POST data. The server will use the given URL, execute some functionality on the server (probably this includes accessing the database) and respond with a view that you can see in your browser.

05 Tests

Every time you run the tests (like described in the chapter Setup), an additional test database gets created. After the tests are done, it gets deleted. It is not connected to the database you use in Swagger. Therefore, we need to populate it for testing new functionality.

The common structure of the tests is:

  • Every class in the models.py, api.py and filters.py should have a corresponding class for testing (e.g. the class FeedbackViewSet in the file api.py gets tested in the class FeedbackTest in the file test–feedback–api.py)
  • The test class begins with a setUpClass. Here the database gets populated. Therefore, you can use: a) a Factory (like the member in FeedbackTest which gets created in the UserFactory) or b) you create the needed objects directly with querysets
  • Every test case should have its own function below SetUpClass to make bug fixing in the future easier

In the project, there are 2 types of tests:

1. Integration tests - test not only one class/function in a file but whole functionality of one part of the project. The server is taken as a 'black box', that is, we do not check what it actually does internally. We only verify that it creates an HTTP response that matches our expectations.

2. Unit tests - test only one function/class, e.g. the model Feedback is tested in test-models.py.

On-site ("bell") notifications

This page is about notifications displayed within the app. If you are interested in push notifications, you may want to read this instead.

On-site notifications are supposed to inform the user about important events, for example when somebody joined their group or they have an activity soon.

Notifications are generated by the backend, either as follow-up events to user actions (user joins a group) or because time passed (activity happens in some hours).

The frontend loads these via API on startup. If there's unseen notifications, it displays the number of unseen notifications.

To add a new notification, have a look at the notifications module on the backend. Create a new signal receiver if a notification depends on a model change, or create a huey crontab task if it depends on time. Make sure to prevent duplicated notifications. If you add more entries to the context JSON field, make sure to follow the existing pattern. You can add an expiry date, too.

Now the notification needs to get integrated in the frontend. The module is called notifications, too. Every notification needs a nice telling text, a route and an icon.

Sparkpost

Document state: provisional, may contain errors or incomplete information. Should be reviewed by Tilmann too.

We use sparkpost for sending emails and accepting inbound emails.

There are quite a few things to consider and this page hopefully will guide you to understand:

  • what to create in the Sparkpost admin interface
  • which DNS records you need to set
  • which variables need to be configured in the ansible config

Sparkpost admin interface

Firstly, you need an account. If you don't already have one ask Nick or Tilmann for an invite. If you don't know who those people are then I think setting up Sparkpost is not your first step ;)

Each deployment of Karrot needs to have the following things:

  • a subaccount
  • an API key for that subaccount (normally created when you create the subaccount)
  • an API key for the main account with two permissions enabled:
    • Inbound Domains: Read/Write
    • Relay Webhooks: Read/Write
  • a sending domain

Keep hold of those two API keys as you'll need to store them as ansible secrets.

Later, during the ansible deployment a webhook and a relay setup (inbound email configuration) will be created for you.

DNS records

When you create a sending domain in Sparkpost, it will tell you which DNS record to add. You want to add the TXT one to verify domain ownership.

I'm not sure if we need to also configure the bounce domain bit.

You also need to setup the inbound email domain for "reply to this email" functionality. By convention we use the main application domain prepended with "replies." e.g. karrot.world -> replies.karrot.world.

For this domain you need to give it some MX records so they will arrive at Sparkpost. See the Sparkpost docs for which records to add.

Note: when we did this using NameCheap it seemed to exclude the subdomain from the wildcard A record we had, requiring a seperate A record for the subdomain, so check your settings!

Ansible configuration

You need the following variables in your secrets.vars.yml:

sparkpost_account_key: <main account API key from above>
sparkpost_subaccount_key: <subaccount API key from above>
sparkpost_relay_secret: <generate a 40 char random string>
sparkpost_webhook_secret: <generate a 40 char random string>

And the following somewhere in the vars for your setup.playbook.yml:

sparkpost_relay_domain: replies.your.domain

Troubleshooting / Tips

The ordering can be tricky as you cannot create a webhook if the app is not deployed at a version that has the webhook endpoint available. There may be some manual deployment steps needed if you are in an intermediate state.

Mobile

We use cordova to package the app as an android app. This gives us access to push notifications (doesn't yet have decent mobile browser support), and offers the installable app experience that some people expect.

Android SDK Setup

Firstly, make sure you have a working Android SDK setup, and a working emulator configuration. See Android Setup for more details.

Get some credentials to make push messaging work

We use google firebase messaging for push notifications. We have two firebase projects, karrot (for karrot.world), and karrot-dev (for dev.karrot.world). If you want access to either of them then ask us, otherwise you can create your own project to use locally.

frontend

  • go to https://console.firebase.google.com
  • select (or create) the project
  • project settings
  • general tab
  • select (or create) the app
  • there is a button to download google-services.json
  • put that in the root of the cordova directory (karrot-frontend/cordova)

backend

  • go to https://console.firebase.google.com
  • project settings
  • cloud messaging tab
  • in the "Project credentials" section there should be a "server key" available
  • put it in your backend local settings under FCM_SERVER_KEY (karrot-backend/config/local_settings.py)

Install

Install cordova globally:

yarn global add cordova

Build the main project:

yarn build:cordova:<config>

where <config> is dev or prod, depending on whether you want to build the dev or the production version.

(You can customize the backend by setting the BACKEND env variable, e.g. in .env)

Enter the cordova directory from the project root:

cd cordova

Add android platform:

cordova platform add android

Run!

./run android dev release <password>

... where <password> is a common yunity password. Ask in chat.karrot.world/channel/karrot-dev if you'd like to know it.

If you have an android device connected in USB debugging mode it will install it on that, otherwise it will try and start an emulator, but you might have to run one yourself if the default does not work.

Then open the chrome developer tools, which can connect to the remote webview:

chrome://inspect

Building

We try and make as easy as possible to build an installable app during development. There are a few pieces:

  • karrot.jks keystore with a common yunity password
  • google-services.json for development (which seems to be safe into include)
  • config.xml containing app details

We maintain a dev config inside the repo. When we are ready for production we will probably keep the files more secret.

The aim is that the dev version would connect to dev.karrot.world and the production one to karrot.world.

Debug build

Use our wrapper script

./build android dev release <password>

... where <password> is a common yunity password. Ask in chat.karrot.world/channel/karrot-dev if you'd like to know it.

If you connect your phone to your computer, you can use the chrome debugging tools with this.

Release build

./build android prod release <release-password>

Android Setup

Android dev stuff is a bit annoying, you need to get an android sdk from somewhere. The easiest approach is to install Android Studio.

Then you need to install sdk platforms/images/apis and one or more emulators. You can do it via the Android Studio GUI, or command line.

Arch Linux

If using Arch Linux there is a nice page about it.

In summary, install these AUR packages:

android-platform-27 android-sdk android-sdk-build-tools android-sdk-platform-tools

Then setup env/path stuff:

export ANDROID_HOME=/opt/android-sdk
export ANDROID_SDK_ROOT=$ANDROID_HOME
export PATH=$ANDROID_HOME/tools:$PATH
export PATH=$ANDROID_HOME/tools/bin:$PATH
export PATH=$ANDROID_HOME/platform-tools:$PATH

It's a good idea to use an sdkusers group to own the /opt/android-sdk files (follow the instructions on the wiki page linked above).

Then install some sdk stuff:

sdkmanager "system-images;android-25;google_apis;x86"

(Update every now and then with sdkmanager --update)

See https://developer.android.com/studio/releases/platforms.html for some more info.

Emulator

Create an emulator, e.g.:

avdmanager create avd --device "Nexus 6P" --name FOO --package "system-images;android-25;google_apis;x86"

Start the emulator:

$(which emulator) -avd FOO -use-system-libs -gpu host -skin 1440x2560