Using Nextjs Image with Directus Assets API

I have read two docs using Nextjs and Directus API and saw one uses the default html img tag and the image scaling is handled by Directus while the other blog uses the Image component from next/image. I tried the first approach using img tags but have been encountering a lot of layout shifts. But when I use next/image Image component, I can set a default width and height using the metadata from the directus_image object. This prevents the layout shift but the issue is image optimisation is done twice, first by Directus API and then by Nextjs. The images are stored in DigitalOcean spaces. Is there a better approach to this issue? TIA.

The code I am using for image loading by directus:

import type { ImageLoaderProps } from "next/image";

interface DirectusImageProps {
  fit?: "cover" | "contain" | "inside" | "outside";
  height?: number;
}

/**
 * Generates a URL for loading images from Directus with specified parameters.
 *
 * @param {ImageLoaderProps & DirectusImageProps} params - The parameters for loading the image.
 * @param {string} params.src - The ID of the image in Directus.
 * @param {number} params.width - The desired width of the image.
 * @param {number} [params.height] - The desired height of the image.
 * @param {number} [params.quality] - The desired quality of the image (default is 75).
 * @param {'cover' | 'contain' | 'inside' | 'outside'} [params.fit] - The fit mode for the image (default is 'contain').
 * @returns {string} The generated URL for the image with the specified parameters.
 */
export default function directusImageLoader({
  src: imageID,
  width,
  quality,
  height,
  fit,
}: ImageLoaderProps & DirectusImageProps): string {
  const url = new URL(`${API_URL}/assets/${imageID}`);
  url.searchParams.set("fit", fit ?? "contain");
  url.searchParams.set("width", (width ?? 800).toString());
  url.searchParams.set(
    "height", height.toString(),
  );
  url.searchParams.set("quality", quality ? quality.toString() : "75");
  return url.href;
}

In a component using next/image:

<Image
    src={directusImageLoader({
       src: material.Images?.[0]?.directus_files_id ?? "",
       width: 400,
       height: height,
       fit: "cover",
       quality: 100,
     })}
     //have to define width and height again to prevent layout shift
     imageWidth={400}
     imageHeight={height}
     imageAlt={`Image of product item`}
>