docs: write up some engine docs

documentation/topics/aggregates.md

@@ -15,7 +15,7 @@ end
 See the documentation for the aggregates section in `Ash.Resource.Dsl.aggregates/1` for more information.
 
 The aggregates declared on a resource allow for declaring a set of named aggregates that can be used by extensions.
-They can also be loaded in the query using `Ash.Query.load/2`, or after the fact using `c:Ash.Api.load/2`. Aggregates declared on the resource will be keys in the resource's struct.
+They can also be loaded in the query using `Ash.Query.load/2`, or after the fact using `c:Ash.Api.load/3`. Aggregates declared on the resource will be keys in the resource's struct.
 
 ## Custom aggregates in the query
 

documentation/topics/calculations.md

@@ -44,7 +44,7 @@ end
 See the documentation for the calculations section in `Ash.Resource.Dsl.calculations/1` for more information.
 
 The calculations declared on a resource allow for declaring a set of named calculations that can be used by extensions.
-They can also be loaded in the query using `Ash.Query.load/2`, or after the fact using `c:Ash.Api.load/2`. Calculations declared on the resource will be keys in the resource's struct.
+They can also be loaded in the query using `Ash.Query.load/2`, or after the fact using `c:Ash.Api.load/3`. Calculations declared on the resource will be keys in the resource's struct.
 
 ## Custom calculations in the query
 

lib/ash/engine/engine.ex

@@ -1,5 +1,38 @@
 defmodule Ash.Engine do
-  @moduledoc false
+  @moduledoc """
+  The Ash engine handles the parallelization/running of requests to Ash.
+
+  Much of the complexity of this doesn't come into play for simple requests.
+  The way it works is that it accepts a list of `Ash.Engine.Request` structs.
+  Some of values on those structs will be instances of `Ash.Engine.Request.UnresolvedField`.
+  These unresolved fields can express a dependence on the field values from other requests.
+  This allows the engine to wait on executing some code until it has its required inputs,
+  or if all of its dependencies are met, it can execute it immediately. The engine's job is
+  to resolve its unresolved fields in the proper order, potentially in parallel.
+  It also has knowledge baked in about certain special fields, like `data` which is the
+  field we are ultimately trying to resolve, and `query` which is the field that drives authorization
+  for read requests. Authorization is done on a *per engine request* basis.
+
+  As the complexity of a system grows, it becomes very difficult to write code that
+  is both imperative and performant. This is especially true of a framework that is
+  designed to be configurable. What exactly is done, as well as the order it is done in,
+  and wether or not is can be parallelized, varies wildly based on factors like how
+  the resources are configured and what capabilities the datalayer has. By implementing
+  a generic "parallel engine", we can let the engine solve for the optimization. We simply
+  have to express the various operations that must happen, and what other pieces of data
+  they need in order to happen, and the engine handles the rest.
+
+  Eventually, this module may (potentially) be used more explicitly, as a way to construct
+  "sagas" or "multis" which represent a series of resource actions with linked up inputs.
+  If each of those resource actions can be broken into its component requests, and the full
+  set of requests can be processed, we can compose large series' of resource actions without
+  having to figure out the most optimal way to do it. They will be done as fast as possible.
+  But we have a long way to go before we get there.
+
+  Check out the docs for `Ash.Engine.Request` for some more information. This is a private
+  interface at the moment, though, so this documentation is just here to explain how it works
+  it is not intended to give you enough information to use the engine directly.
+  """
   defstruct [
     :api,
     :requests,

lib/ash/engine/request.ex

@@ -1,14 +1,22 @@
 defmodule Ash.Engine.Request do
-  @moduledoc false
+  @moduledoc """
+  Represents an individual request to be processed by the engine.
+
+  See `new/1` for more information
+  """
 
   alias Ash.Error.Forbidden.MustPassStrictCheck
   alias Ash.Error.Framework.AssumptionFailed
   alias Ash.Error.Invalid.{DuplicatedPath, ImpossiblePath}
 
   defmodule UnresolvedField do
-    @moduledoc false
+    @moduledoc """
+    Represents an unresolved field to be resolved by the engine
+    """
     defstruct [:resolver, deps: [], data?: false]
 
+    @type t :: %__MODULE__{}
+
     def new(dependencies, func) do
       %__MODULE__{
         resolver: func,
@@ -69,10 +77,63 @@ defmodule Ash.Engine.Request do
 
   alias Ash.Authorizer
 
+  @doc """
+  Create an unresolved field.
+
+  Can have dependencies, which is a list of atoms. All elements
+  before the last comprise the path of a request that is also
+  being processed, like `[:data]`, and the last element is the
+  key of that request that is required. Make sure to pass a
+  list of lists of atoms. The second argument is a map, which
+  contains all the values you requested, at the same path
+  that they were requested.
+
+  For example:
+
+      resolve([[:data, :query], [:data, :data]], fn %{data: %{query: query, data: data}} ->
+        data # This is the data field of the [:data] request
+        query # This is the query field of the [:data] request
+      end)
+  """
+  @spec resolve([[atom]], (map -> {:ok, term} | {:error, term})) :: UnresolvedField.t()
   def resolve(dependencies \\ [], func) do
     UnresolvedField.new(dependencies, func)
   end
 
+  @doc """
+  Creates a new request.
+
+  The field values may be explicit values, or they may be
+  instances of `UnresolvedField`.
+
+  When other requests depend on a value from this request, they will
+  not be sent unless this request has completed its authorization (or this
+  request has been configured not to do authorization). This allows requests
+  to depend on eachother without those requests happening just before a request
+  fails with a forbidden error. These fields are `data`, `query`, `changeset`
+  and `authorized?`.
+
+  A field may not be resolved  if the data of a request has been resolved and
+  no other requests depend on that field.
+
+  Options:
+
+    * query - The query to be used to fetch data. Used to authorize reads.
+    * data - The ultimate goal of a request is to compute the data
+    * resource - The primary resource of the request. Used for openeing transactions on creates/updates/destroys
+    * changeset - Any changes to be made to the resource. Used to authorize writes.
+    * path - The path of the request. This serves as a unique id, and is the way that other requests can refer to this one
+    * action_type - The action_type of the request
+    * action - The action being performed on the data
+    * async? - Whether or not the request *can* be asynchronous, defaults to `true`.
+    * api - The api module being called
+    * name - A human readable name for the request, used when logging/in errors
+    * strict_check_only? - If true, authorization will not be allowed to proceed to a runtime check (so it cannot run db queries unless authorization is assured)
+    * actor - The actor performing the action, used for authorization
+    * authorize? - Wether or not to perform authorization (defaults to true)
+    * verbose? - print informational logs (warning, this will be a whole lot of logs)
+    * write_to_data? - If set to false, this value is not returned from the initial call to the engine
+  """
   def new(opts) do
     query =
       case opts[:query] do
@@ -596,8 +657,8 @@ defmodule Ash.Engine.Request do
   defp try_resolve_local(request, field, internal?) do
     authorized? = Enum.all?(Map.values(request.authorizer_state), &(&1 == :authorized))
 
-    # Don't fetch honor requests for dat until the request is authorized
+    # Don't fetch honor requests for data until the request is authorized
-    if field in [:data, :query, :changeset] and not authorized? and not internal? do
+    if field in [:data, :query, :changeset, :authorized?] and not authorized? and not internal? do
       try_resolve_dependencies_of(request, field, internal?)
     else
       case Map.get(request, field) do

mix.exs

@@ -83,6 +83,9 @@ defmodule Ash.MixProject do
         filter: ~r/Ash.Filter/,
         "resource introspection": ~r/Ash.Resource/,
         "api introspection": ~r/Ash.Api/,
+        engine: [
+          ~r/Ash.Engine/
+        ],
         miscellaneous: [
           Ash.NotLoaded,
           Ash.Error.Stacktrace,