I18n within resources

davidmccrea
2023-02-07

davidmccrea:

Hi!

Is there a way to replicate the functionality of something like Trans ( https://hexdocs.pm/trans/Trans.html ) in Ash? It provides a simple way to translate fields of a schema without needing extra db tables and joins.

What is the idomatic way to handle I18n within resources in Ash?

zachdaniel:

Hey there! There is no set pattern for it currently, but you can do it with calculations, for example

zachdaniel:

If you wanted the translations stored in the database you could basically do exactly what that tool does

zachdaniel:

and define an embedded attribute

zachdaniel:

defmodule MyApp.Post.Translations do
  use Ash.Resource,
    data_layer: :embedded

  attributes do
    attribute :en, MyApp.Post.Translation
  end
end

defmodule MyApp.Post.Translation do
  ...

  attributes do
    attribute :translated_field, :field
  end
end

defmodule MyApp.Post do
  use Ash.Resource

  attributes do
    attribute :translations, MyApp.Post.Translations
  end
end

themykolas:

Could we possibly have a working example, even if you point something out.

This looks like an extension to me

where you would write:

attributes do
  attribute :slug, :translated_field, :unique
  attribute :title, :translated_field
  attribute :description, :translated_field
do

I’m building a page builder and have blocks not sure how would i write out the way you’ve described for each block? Is this possible, could you point me in the right direction?

themykolas:

which would result in this structure in db

translations: {
    en: { slug: "asdf, title: "ASdfa" , decription: "adfs" },
    it: { slug: "asdf, title: "ASdfa" , decription: "adfs" }
    ... and so forth for all condifgured locales
}

zachdaniel:

Hey there, sorry about that

zachdaniel:

Been a bit busy lately 🙂

zachdaniel:

My initial example was a bit off

zachdaniel:

Actually…maybe not so much 🙂

zachdaniel:

It groups translations effectively the way you want

zachdaniel:

defmodule MyApp.Post.Translations do
  use Ash.Resource,
    data_layer: :embedded

  attributes do
    attribute :en, MyApp.Post.Translation
  end
end

defmodule MyApp.Post.Translation do
  ...

  attributes do
    attribute :name, :string
  end
end

defmodule MyApp.Post do
  use Ash.Resource

  attributes do
    attribute :name, :string
    attribute :translations, MyApp.Post.Translations
  end
end

zachdaniel:

I updated the example to show roughly how it would look with a real field

zachdaniel:

So with that you’d have a record like:

name: "foo", # this is optional, but lets you have like a primary value (i.e maybe this is `en`)
translations: %MyApp.Post.Translations{
  en: %MyAPp.Post.Translation{
    name: "oof"
  }
}

zachdaniel:

So then what you’d do is add all the language code as attributes

zachdaniel:

defmodule MyApp.Post.Translations do
  use Ash.Resource,
    data_layer: :embedded

  attributes do
    for language_code <- ~w(en jp the_rest)a do
      attribute language_code, MyApp.Post.Translation
    end
  end
end

themykolas:

Awesome, thank you, so this means i need to define all the modules by hand to have this work properly. Do you imagine a way i could write an extension or macro to do this automatically?

so in the end the resource would look like:

defmodule MyApp.Post do
  use Ash.Resource

  attributes do
    i18n_attribute :slug, :string, required: true, unique: true
    i18n_attribute :name, :string

  end
end

zachdaniel:

I see what you mean. Yes that is possible. Not in the way you’ve shown, but it could be done with a separate DSL section.

i18n_attributes [:slug, :name]

themykolas:

ahh that’s nice!

themykolas:

I’m currently thinking about migrating my octafest.com from the old laravel php stack to phoenix liveview and ash seems like the perfect bridge

zachdaniel:

well if you’d like to work on ash_i18n I’m more than happy to help 😄

themykolas:

where would i start?

zachdaniel:

Step one would be to get familiar with spark dsl extensions. Not a lot of guides out there, but a great example of a lightweight extension is ash_archival

themykolas:

ok on it

zachdaniel:

The idea would be to add a dsl extension to describe the changes, and then to use transformers to apply changes the resource

themykolas:

where can i find a simple example implementing DSL.Entity

themykolas:

as i want to do this:

defmodule AshI18n.Resource do
  alias AshI18n.I18nField

  @i18n_attribute_schema [
    name: [
      type: :atom,
      required: true,
      doc: "The name of the field"
    ],
    validations: [
      type:  :keyword_list,
      doc: "The basic validations for the field",
      default: [],
      keys: [required: [type: :atom], unique: [type: :atom]]
    ]
  ]

  @i18n_attribute %Spark.Dsl.Entity{
    name: :i18n_attribute,
    describe: "Adds a translated field",
    examples: [
      "field :slug, :required, :unique",
      "field :title"
    ],
    target: I18nField,
    args: [:name, :validations],
    auto_set_fields: true,
    schema: @i18n_attribute_schema
  }

  @i18n_attributes %Spark.Dsl.Section{
    name: :i18n_attributes,
    describe: "A section for configuring translations for a resource.",
    examples: [
      """
      i18n_attributes do
        field :slug, [:required, :unique]
        field :title, []
      end
      """
    ],
    entities: [
      @i18n_attribute
    ]
  }

  use Spark.Dsl.Extension,
    sections: [@i18n_attributes],
    transformers: [AshI18n.Resource.Transformers.SetupI18n]
end

themykolas:

here’s my field:

defmodule AshI18n.I18nField do
  defstruct name: "", validations: []

  def i18n_attributes(resource) do
    Spark.Dsl.Extension.get_entities(resource, [:i18n_attributes])
  end
end

themykolas:

but this fails with

❯ mix phx.server
Compiling 4 files (.ex)

00:50:06.367 [error] Task #PID<0.303.0> started from #PID<0.287.0> terminating
** (FunctionClauseError) no function clause matching in Keyword.merge/2
    (elixir 1.14.5) lib/keyword.ex:979: Keyword.merge([], true)
    (spark 1.1.11) lib/spark/dsl/extension.ex:1078: anonymous fn/9 in Spark.Dsl.Extension.do_build_section/6
    (elixir 1.14.5) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
    (spark 1.1.11) lib/spark/dsl/extension.ex:1075: Spark.Dsl.Extension.do_build_section/6
    lib/ash_i18n/resource.ex:48: anonymous fn/3 in :elixir_compiler_3.__MODULE__/1
    (elixir 1.14.5) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.14.5) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 4.3.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Function: #Function<0.104469677/0 in Kernel.ParallelCompiler.async/1>
    Args: []

== Compilation error in file lib/ash_i18n/resource.ex ==
** (exit) an exception was raised:
    ** (FunctionClauseError) no function clause matching in Keyword.merge/2
        (elixir 1.14.5) lib/keyword.ex:979: Keyword.merge([], true)
        (spark 1.1.11) lib/spark/dsl/extension.ex:1078: anonymous fn/9 in Spark.Dsl.Extension.do_build_section/6
        (elixir 1.14.5) lib/enum.ex:2468: Enum."-reduce/3-lists^foldl/2-0-"/3
        (spark 1.1.11) lib/spark/dsl/extension.ex:1075: Spark.Dsl.Extension.do_build_section/6
        lib/ash_i18n/resource.ex:48: anonymous fn/3 in :elixir_compiler_3.__MODULE__/1
        (elixir 1.14.5) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
        (elixir 1.14.5) lib/task/supervised.ex:34: Task.Supervised.reply/4
        (stdlib 4.3.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

zachdaniel:

auto_set_fields is meant to be a keyword list of explicit field values

zachdaniel:

you probably don’t need it 🙂

themykolas:

ok got this to build 😄


  i18n_attributes do
    i18n_attribute(:slug, :string, constraints: [required: true, unique: true])
    i18n_attribute(:title, :string)
  end

themykolas:

now on to figuring out how to embed this, any pointers?

themykolas:

Does this look like i’m moving in the right direction?

themykolas:

Or is there a better/cleaner way to insert embeded schemas?

themykolas:

I’m thinking if it’s worth the effort it’s not that hard to do what you’ve shown and this adds quite a bit of magic that you can’t see.

themykolas:

because this is just for adding the translated fields. I then need to modify the create changeset to include the embeds with validations add read actions which accept locale and move the fields into top level access

\ ឵឵឵:

It really depends how you want to do this:

Is the translation template static for all instances of the resource? In this case, you’re better off using something like Gettext (included by default with Phoenix) and then returning the interpolated string using a calculation.

Are the translations something that need to be modifiable from a frontend, i.e. they should live in the database and not in code? Then there are a lot of options, and which is best still depends on whether individual resources need to have arbitrary strings or there is a single template for each locale that needs to be user-modifiable but applies to all instances of that resource.

zachdaniel:

Yeah, if you’re looking to store translations in the database then that is a strategy that would work, and defining the module like that is pretty much the only way

themykolas:

Yeah the whole point is to have client facing editable content for pages etc.

themykolas:

I can’t get this to render the form though i’ve tried adding forms: [auto?: true] but nothing happens:

        <.simple_form for={@create_form} phx-submit="create_post">
          <.input field={@create_form[:title]} type="text" placeholder="Title..." />
          <.input field={@create_form[:content]} type="textarea" placeholder="Content..." />

          <.inputs_for :let={translations} field={@create_form[:translations]}>
            <.inputs_for :let={translation} :for={locale <- @locales} field={translations[locale]}>
              

<%= locale %>

<.input field={translation[:title]} type="text" /> <.input field={translation[:content]} type="text" />
<.button> create

themykolas:

Any help is greatly appreciated.

zachdaniel:

You likely need to do an add_form when you create the initial form

zachdaniel:

because we don’t assume that a value should be populated for translations by default

zachdaniel:

I can look at it a bit more this weekend, but if you want an empty value provided for a given thing you’d need to add_form once at the beginning

zachdaniel:

and then likely for each locale translation you could have a little plus button that adds an additional locale translation

themykolas:

It should have an empty value for each locale as i have a tabs thing where you can switch between languages. I’ll have a look at the add_form thing 🙂 thank you! Ash is amazing 🙂

themykolas:

Hmm ok back at this, i have an interesting case and trying to figure out how to best do this:

Octafest is a multi-tenant thing. And translations are really tenant bound and dynamic. For e.g. Tenant A has two languages English and Lithuanian. Default is Lithuanian. Tenant B has three languages English, Italian and German. Default is english.

Since json columns don’t really need a defined structure the way i’ve done it in php land is just have json columns for the translated stuff and then show the ui according to the tenants settings.

Tenant settings define what languages they support and the default language. That in turn determines what get’s populated in the translated field column form.

Also i’ve switched out the relationship, so it’s not:

model 
-> translations 
  -> [
       en -> [ name: 'English name', description: 'English description ],
       lt -> [ name: 'Lithuanian name', description: 'Lithuanian description' ]       
]

But:

model -> name -> [en: 'English name', lt: 'Lithuanian name']
      -> description -> [en: 'English description', lt: 'Lithuanian description']

Any help is greatly appreciated

themykolas:

Ok so created a shared embed like so:

defmodule Octafest.Shared.Translated do
  @behaviour Access

  use Ash.Resource,
  data_layer: :embedded

  attributes do
    for locale <- ~w(en lt)a do
      attribute locale, :string
    end
  end

  @impl Access
  def fetch(term, key), do: Map.fetch(term, key)

  @impl Access
  def get_and_update(data, key, func) do
    Map.get_and_update(data, key, func)
  end

  @impl Access
  def pop(data, key), do: Map.pop(data, key)

  @impl Access
  def get(map, key, default), do: Map.get(map, key, default)

end

But the problem is still having the hardcoded locales. I mean as a workaround i can have that as a list of all available locales which for now is limited and then only show the relevant ones for the tenant?

themykolas:

This is the usage:

defmodule Octafest.Blog.Post do
  use Ash.Resource,
    data_layer: AshPostgres.DataLayer

  postgres do
    table "posts"

    repo Octafest.Repo
  end

  code_interface do
    define_for Octafest.Blog
    define :create, action: :create
    define :read_all, action: :read
    define :update, action: :update
    define :destroy, action: :destroy
    define :get_by_id, args: [:id], action: :by_id
  end

  actions do
    defaults ~w(create read update destroy)a

    read :by_id do
      argument :id, :uuid, allow_nil?: false
      get? true

      filter expr(id == ^arg(:id))
    end
  end

  attributes do
    uuid_primary_key :id

    attribute :title, Octafest.Shared.Translated
    attribute :content, Octafest.Shared.Translated
  end
end

themykolas:

And this is how a basic form for creating looks like:

zachdaniel:

You strategy of supporting all locales would work

zachdaniel:

You can also make it a calculation that produces a map, and then use a custom type in ash_graphql .

zachdaniel:

oh, you might not even be using ash_graphql ?

zachdaniel:

anyway, a custom map type could also do it 🙂