.sevenseacat:

My app has a resource called Vendor for interacting with external vendors, so we store user credentials for those vendors encrypted in the resource. I’ve now added a validation when creating vendors, that validates that the credentials are actually valid (by talking to the external API). This works great!

However, I have a test suite that generates lots of Vendor resources in the course of testing things, and 99% of them don’t use valid credentials (nor should they!) so now all the tests are failing due to this new validation.

I can see a couple of options:

• In my test functions, skip validation when calling eg. Vendor.create(data) . This would be the easiest way, but I can’t see any way in Ash to do it? (I thought there might be a nice validate?: false opt like there’s a authorize?: false opt but there isn’t)

• Use Mimic ( https://github.com/edgurgel/mimic ) (already used in the app) to globally stub out the new validation module, to bypass it. I can foresee running into issues when I want to actually test the validation though - I’d have to muck around with unstubbing/restubbing and I’m not sure if Mimic even supports it.

• Create a new action in my resource that is the same as the create action but doesn’t include the extra validation. This would be my least favorite option.

• Actually my least favorite option (but still an option) would be to use valid credentials to test accounts in all of the tests, but I’m not even considering doing this.

Have I missed something in the Ash docs on how to skip validations, or how would other people approach doing this?

.sevenseacat:

I have just seen that its possible to skip_global_validations? from inside an action (I’m guessing that would be validations from a validations block on the resource?) so there might be a way to skip validations from outside, hidden somewhere

kernel_io:

validation module config which always returns valid = true when run in test

kernel_io:

done via dev.exs, test.exs 🤷🏿‍♂️

kernel_io:

even easier is swapping out the valid? function at compile time

zachdaniel:

Yeah, there are a couple options along those lines. You could also look for something like context.skip_credential_validation , and then set the context when creating things in test.

zachdaniel:

Because validations and changes aren’t named, we can’t provide an elegant way to skip individual ones in test, and an option to skip all of them with an option when creating is only useful in very specific cases. We could add a name option to changes and validations, and then add skip: [:foo, :bar] ?

.sevenseacat:

Yeah I thought later maybe I could add an argument to the action to decide whether or not to run the external validation, defaulting to true, set to false in tests

zachdaniel:

You can also make it a private? argument if you don’t want it to be settable as user input

zachdaniel:

private arguments have to be set explicitly with Ash.Changeset.set_argument/2

.sevenseacat:

that would be doable, given I only do it a few places in factory methods I think

.sevenseacat:

I’ll take a look at doing this shortly and mark resolved if I can get it all working 🙂

.sevenseacat:

I think I’m not understanding how to run the validation conditionally. I’ve tried:

create :create do
      argument :run_external_validations, :boolean, default: true, private?: true

      validate {Vendor.CredentialValidation, []},
        only_when_valid?: true,
        before_action?: true,
        where: &Ash.Changeset.get_argument(&1, :run_external_validations)

but that seems to always skip it

zachdaniel:

interesting

.sevenseacat:

the docs say

Validations that should pass in order for this validation to apply. These validations failing will not invalidate the changes, but will instead result in this validation being ignored. Accepts a module, module and opts, or a 1 argument function that takes the changeset. Defaults to [] .

so its not “return a boolean to run”, but I don’t know what it is

zachdaniel:

yeah, I feel like you should be getting an error

zachdaniel:

The basic thing is that where is supposd to also be a validation

zachdaniel:

and so should return :ok | {:error, error}

zachdaniel:

which we use essentially as a boolean flag

zachdaniel:

but allows composing validations.

.sevenseacat:

okay so I’m definitely using it very wrong

zachdaniel:

there isn’t an argument_equals builtin validation right now (we should add one though)

.sevenseacat:

I can add one in my project and then contribute it back 👍

zachdaniel:

you can probably get a jump start by copying the built in attribute_equals.ex

.sevenseacat:

I got this to work 😄

My resource code:

    create :create do
      primary? true
      argument :run_external_validations, :boolean, default: true, private?: true

      validate {Vendor.CredentialValidation, []},
        only_when_valid?: true,
        before_action?: true,
        where: argument_equals(:run_external_validations, true)

My test factory code:

  defp insert_vendor_params!(params) do
    Vendor
    |> Changeset.new(params)
    |> Changeset.set_argument(:run_external_validations, false)
    |> Changeset.for_create(:create, authorize?: false)
    |> MyApp.Api.create!()
  end