Manifest extensibility enhancements: support additive mutations instead of full override or ignore with ManifestPublishingContext / ManifestPublishingCallbackAnnotation

[tdhatcher]

Problem

I'm trying to use the manifest of an Aspire model as input to drive some proprietary CLI tooling to further simplify onboarding for teams. A mono repository in Git is not an option to store all source code for dependencies. So ideally a newcomer to the project would only have to have their dev machine setup with the CLI tool, clone the Aspire project and simply run a CLI command to get up an running.

Idea

Our CLI's magic will then read the manifest for metadata and handle cloning all Git repositories to a relative location the Aspire project expects.

This eliminates the need for written documentation that must be maintained about which repositories need to be manually cloned to successfully run our Aspire project and allows us to fully automate that process making onboarding even more "turn key".

The Aspire app model might describe this metadata like

var anotherService = builder
  .AddProject<Projects.AnotherService>("anotherService")
  .WithGitRepository("https://dev.azure.com/mycompany/_git/AnotherService")
  ...

At this point the addition of the WithGitRepository extension will invoke WithManifestPublishingCallback

public static IResourceBuilder<T> WithGitRepository<T>(this IResourceBuilder<T> builder, string repositoryUrl)
    where T : ProjectResource
{
    builder.WithAnnotation(new GitRepositoryAnnotation(repositoryUrl));
    builder.WithManifestPublishingCallback(context => {
        context.Writer.WriteStartObject("repository");
        context.Writer.WriteString("url", repositoryUrl);
        context.Writer.WriteEndObject();
    });
    return builder;
}

A few issues arise around extensibility:

1. The ManifestPublishCallbackAnnotation lacks a way to express whether its mutation intends to fully override, be ignored, or just be additive. In my case I wish to make it additive, adding a new object key that's it.
2. Incomplete context: Adding a ManifestPublishCallbackAnnotation using the WithManifestPublishingCallback does not provide a reference to the resource or access to the default methods on the ManifestPublishingContext to delegate back to. I have no recourse but to rewrite my own so it's also lacking exposure of necessary details. My preference would be to avoid reimplementing any of the default implementation and have to maintain parity as time goes on. This could be made more flexible and not be exclusive as full override or not.

   aspire/src/Aspire.Hosting/Publishing/ManifestPublishingContext.cs

   Lines 88 to 130 in 7f63747

   	internal async Task WriteResourceAsync(IResource resource)
   	{
   	// First see if the resource has a callback annotation with overrides the behavior for rendering
   	// out the JSON. If so use that callback, otherwise use the fallback logic that we have.
   	if (resource.TryGetLastAnnotation<ManifestPublishingCallbackAnnotation>(out var manifestPublishingCallbackAnnotation))
   	{
   	if (manifestPublishingCallbackAnnotation.Callback != null)
   	{
   	await WriteResourceObjectAsync(resource, () => manifestPublishingCallbackAnnotation.Callback(this)).ConfigureAwait(false);
   	}
   	}
   	else if (resource is ContainerResource container)
   	{
   	await WriteResourceObjectAsync(container, () => WriteContainerAsync(container)).ConfigureAwait(false);
   	}
   	else if (resource is ProjectResource project)
   	{
   	await WriteResourceObjectAsync(project, () => WriteProjectAsync(project)).ConfigureAwait(false);
   	}
   	else if (resource is ExecutableResource executable)
   	{
   	await WriteResourceObjectAsync(executable, () => WriteExecutableAsync(executable)).ConfigureAwait(false);
   	}
   	else if (resource is IResourceWithConnectionString resourceWithConnectionString)
   	{
   	await WriteResourceObjectAsync(resource, () => WriteConnectionStringAsync(resourceWithConnectionString)).ConfigureAwait(false);
   	}
   	else if (resource is ParameterResource parameter)
   	{
   	await WriteResourceObjectAsync(parameter, () => WriteParameterAsync(parameter)).ConfigureAwait(false);
   	}
   	else
   	{
   	await WriteResourceObjectAsync(resource, WriteErrorAsync).ConfigureAwait(false);
   	}
   	async Task WriteResourceObjectAsync<T>(T resource, Func<Task> action) where T : IResource
   	{
   	Writer.WriteStartObject(resource.Name);
   	await action().ConfigureAwait(false);
   	Writer.WriteEndObject();
   	}
   	}

3. If you have multiple extension method calls using WithManifestPublishingCallback all wanting to mutate the manifest in subtle ways on an IResource the last one wins.

Potential areas of enhancement:

1. Provide a property on the ManifestPublishingCallbackAnnotation to indicate whether this should be override or additive?
2. Further enrich the ManifestPublishingContext which invokes the callback registered with the ManifestPublishingCallbackAnnotation to expose more detail about the current context.
3. Provide an additional overload to WithManifestPublishingCallback extensions in which IResource reference to the callback is provided with a signature such as Func<ManifestPublishingContext, IResource, Task>
4. Provide an additional overload to WithManifestPublishingCallback so that your callback can invoke this action delegate rather than the other way around. The context will still create start and end object, but this provides an option to invert control so the callback chooses to invoke it or not if you want the default (similar to how middleware can chose to halt the request pipeline by not calling await _next(context)).
   

   aspire/src/Aspire.Hosting/Publishing/ManifestPublishingContext.cs

   Lines 124 to 128 in 7f63747

   	async Task WriteResourceObjectAsync<T>(T resource, Func<Task> action) where T : IResource
   	{
   	Writer.WriteStartObject(resource.Name);
   	await action().ConfigureAwait(false);
   	Writer.WriteEndObject();

5. Perhaps it's simpler to normalize around ManifestPublishingCallbackAnnotation having some type of property like ManifiestPublishingEvents to follow a flexible eventing pattern which contain numerous Func<ManifestPublishingContext, IResource, Task> would be more appropriate to modify behavior at different points of the "writing" processes. Similar to how OpenIdConnectEvents work. https://github.com/dotnet/aspnetcore/blob/794124fda4e2de1db0e6893dc0cc031a181ce52a/src/Security/Authentication/OpenIdConnect/src/Events/OpenIdConnectEvents.cs#L9-L72

[davidfowl]

This is a big area of investment for the next wave of features. No real details yet but this is issue is good motivation.

Is this a publish time feature you’re building or a runtime one?

[tdhatcher]

@davidfowl publish time. My initial approach was to take the simplest to first get something working. I was envisioning a run once or as needed scenario.

I could see the manifest (or a flavor of it) being an automated commit by a build pipeline to keep it in sync and stored alongside aspire host in source control or stored in some other arbitrary location that dev tooling could easily retrieve.

In my case I find it acceptable and similar dev experience of how a node project has a package.json where a developer with basic skill can be expected to first run npm install followd by occassional invocations as new code additions require.

Unrelated feedback: The company I work at, I have a pretty solid use case for Aspire that has pleny of complexity which covers quite a few areas around an existing customer-facing ecosystem. I took a retrofitting approach with the apsire apphost and made it work without any code modifications to the projects and its all running (including the OTEL observability which I wrapped into a hosting startup assembly). The most complex resource is about 25 lines, mostly environment overrides. Made it work, now making it better and finding ways to improve the experience.

My current goal is to figure out how to automate all of the manual one time setup into more turnkey setup. And have real world working reference for all the other dev teams to model from.

The ecosystem requires redis, elasticsearch, mssql server, external Okta auth interactions, certificates, secrets hosted elsewhere, 6 dotnet services in code bases, and a several external dotnet services will not run but depend on, 4 angular clients requiring 2 incompatible versions of node, multi-branded hostnames. Being able to represent setting up a resource in 5-20 lines really beats have to explain and maintain documentation (which drifts).