esm.sh VS compression-dictionary-transport

/* eslint-disable @typescript-eslint/ban-ts-comment */ // Follow this setup guide to integrate the Deno language server with your editor: // https://deno.land/manual/getting_started/setup_your_environment // This enables autocomplete, go to definition, etc. import { corsHeaders } from "../_shared/cors.ts"; import { createClient } from "https://esm.sh/@supabase/[email protected]"; import randomSample from "https://esm.sh/@stdlib/[email protected]"; import Replicate from "https://esm.sh/[email protected]"; import { base64 } from "https://cdn.jsdelivr.net/gh/hexagon/base64@1/src/base64.js"; const supabaseClient = createClient( Deno.env.get("SUPABASE_URL") ?? "", Deno.env.get("SUPABASE_SERVICE_ROLE_KEY") ?? "" ); const replicate = new Replicate({ auth: Deno.env.get("REPLICATE_API_TOKEN") ?? "", }); // @ts-expect-error Deno.serve(async (req) => { if (req.method === "OPTIONS") { return new Response("ok", { headers: corsHeaders }); } const { record } = await req.json(); const thought_id = record.id; if (!thought_id) { return new Response("Missing thought_id", { status: 400, headers: { "Content-Type": "application/json", ...corsHeaders }, }); } const allObjectIDsResponse = await fetch( "https://collectionapi.metmuseum.org/public/collection/v1/objects?departmentIds=11", { method: "GET", headers: { "Content-Type": "application/json", Accept: "application/json", }, } ); const { objectIDs } = await allObjectIDsResponse.json(); const listOfArtworks = []; const addedIDs: number[] = []; while (listOfArtworks.length < 80) { const randomObjectID = randomSample(objectIDs, { size: 1 })[0]; if (addedIDs.includes(randomObjectID)) continue; const res = await fetch( `https://collectionapi.metmuseum.org/public/collection/v1/objects/${randomObjectID}`, { method: "GET", headers: { "Content-Type": "application/json", Accept: "application/json", }, } ); const artwork = await res.json(); if (!artwork.primaryImageSmall) continue; addedIDs.push(artwork.objectID); listOfArtworks.push({ image_url: artwork.primaryImageSmall, artist_name: artwork.artistDisplayName, title: artwork.title, is_main: listOfArtworks.length === 0, is_variant: false, thought_id, }); } const mainImage = listOfArtworks[0]; const output = await replicate.run( "yorickvp/llava-13b:b5f6212d032508382d61ff00469ddda3e32fd8a0e75dc39d8a4191bb742157fb", { input: { image: mainImage.image_url, top_p: 1, prompt: "Describe this painting by " + mainImage.artist_name, max_tokens: 1024, temperature: 0.2, }, } ); const file = await fetch(mainImage.image_url).then((res) => res.blob()); const promises = []; for (let i = 0; i < 8; i++) { const body = new FormData(); body.append( "prompt", output.join("") + `, a painting by ${mainImage.artist_name}` ); body.append("output_format", "jpeg"); body.append("mode", "image-to-image"); body.append("image", file); body.append("strength", clamp(Math.random(), 0.4, 0.7)); const request = fetch( `${Deno.env.get( "STABLE_DIFFUSION_HOST" )}/v2beta/stable-image/generate/sd3`, { method: "POST", headers: { Accept: "application/json", Authorization: `Bearer ${Deno.env.get("STABLE_DIFFUSION_API_KEY")}`, }, body, } ); promises.push(request); } const results = await Promise.all(promises); const variants = await Promise.all(results.map((res) => res.json())); await supabaseClient.from("artworks").insert(listOfArtworks); for (let i = 0; i < variants.length; i++) { const variant = variants[i]; const randomId = Math.random(); await supabaseClient.storage .from("variants") .upload( `${thought_id}/${randomId}.jpeg`, base64.toArrayBuffer(variant.image), { contentType: "image/jpeg", } ); await supabaseClient.from("artworks").insert({ image_url: `${Deno.env.get( "SUPABASE_URL" )}/storage/v1/object/public/variants/${thought_id}/${randomId}.jpeg`, artist_name: mainImage.artist_name, is_main: false, is_variant: true, thought_id, }); } await supabaseClient .from("thoughts") .update({ generating: false }) .eq("id", thought_id); return new Response(JSON.stringify({ main: mainImage }), { headers: { "Content-Type": "application/json", ...corsHeaders }, }); }); function clamp(value: number, min: number, max: number) { return Math.min(Math.max(value, min), max); }

const extism = await import("https://esm.sh/@extism/extism");

Unpkg serves whatever is published to NPM, and if it's a library intended for the browser, that often includes minified versions ready for use in script tags, for example, https://unpkg.com/[email protected]/mithril.min.js. Sometimes the default export is CJS (which has require() calls), in which case, you can usually use the browse url that I mentioned to see if there's another export you can use.

https://esm.sh/ is definitely a good option too if you're OK with modules.

* Change my NPM imports to something that would work with Deno. The most straightforward thing to do was just change `import "foo"` to `import "npm:foo"`, but this felt hacky so eventually I used https://esm.sh, which worked for some packages but not others.

import { createClient } from "https://esm.sh/@supabase/supabase-js@2"; import { create, Payload } from "https://deno.land/x/[email protected]/mod.ts"; interface User { username: string; pin: string; } const supabase = createClient(Deno.env.get('SUPABASE_URL'), Deno.env.get('SUPABASE_SERVICE_ROLE_KEY')); const handler = async (req: Request): Promise => { if (req.method !== 'POST') { return new Response(null, { status: 405 }); } const { username, pin } = await req.json() as Vehicle; if (!username|| !pin) { return new Response(JSON.stringify({ error: 'username and PIN are required' }), { status: 400 }); } const { data, error } = await supabase .from('users') .select('*') .eq('username', username) .eq('pin', pin) .single(); if (error || !data) { return new Response(JSON.stringify({ valid: false }), { status: 401 }); } const jwtSecret = Deno.env.get('JWT_SECRET') as string;; const payload: Payload = { "username": username }; try { const token = await create({ alg: "HS256", typ: "JWT" }, payload, jwtSecret); // This line has the error console.log(token); return new Response(JSON.stringify({ token }), { status: 200 }); } catch (error) { console.error('Error creating JWT:', error); return new Response(JSON.stringify({ error: 'Error creating JWT' }), { status: 500 }); } }; Deno.serve(handler);

Easily serve libraries from local server/private VPS: You can try to serve and cache libraries by running esm.sh to improve loading times on your server side. Or to keep things simple, just upload a code to pastebin or similar services and directly use it here!

import { serve } from 'https://deno.land/[email protected]/http/server.ts'; import { WebClient } from 'https://deno.land/x/[email protected]/mod.js'; import { SupabaseClient } from 'https://esm.sh/@supabase/supabase-js@2'; const slack_bot_token = Deno.env.get("SLACK_TOKEN") ?? ""; const bot_client = new WebClient(slack_bot_token); const supabase_url = Deno.env.get("SUPABASE_URL") ?? ""; const service_role = Deno.env.get("SUPABASE_SERVICE_ROLE_KEY"); const supabase = new SupabaseClient(supabase_url, service_role); console.log(`Slack URL verification function up and running!`); serve(async (req) => { try { const req_body = await req.json(); console.log(JSON.stringify(req_body, null, 2)); const { token, challenge, type, event } = req_body; if (type == 'url_verification') { return new Response(JSON.stringify({ challenge }), { headers: { 'Content-Type': 'application/json' }, status: 200, }); } else if (event.type == 'app_mention') { const { user, text, channel, ts } = event; const url_path = text.toLowerCase() .includes('code') ? '/code' : '/general'; const { error } = await supabase.from('job_queue').insert({ http_verb: 'POST', payload: { user, text, channel, ts }, url_path: url_path }); if (error) { console.error(error); return new Response(JSON.stringify({ error: error.message }), { headers: { 'Content-Type': 'application/json' }, status: 400, }); } await post(channel, ts, `Taking a look and will get back to you shortly!`); return new Response('', { status: 200 }); } } catch (error) { return new Response(JSON.stringify({ error: error.message }), { headers: { 'Content-Type': 'application/json' }, status: 400, }); } }); async function post(channel: string, thread_ts: string, message: string): Promise { try { const result = await bot_client.chat.postMessage({ channel: channel, thread_ts: thread_ts, text: message, }); console.info(result); } catch (e) { console.error(`Error posting message: ${e}`); } }

> Dictionary entries (or at least the metadata) should be cleared any time cookies are cleared.

So it seems it should not get you anything you cannot already do with cookies.

https://github.com/WICG/compression-dictionary-transport?tab...

Talked about here:

https://github.com/WICG/compression-dictionary-transport

There's a spec up for custom dictionary support across the web. https://github.com/WICG/compression-dictionary-transport

This was one of the major blockers that iirc Mozilla threw in the way of zstd compression support: they said zstd with a standardly accepted dictionary would be too particular & wanted more. With this spec maybe Moz will accept zstd as a web compression standard.

Here here. Today, bundlers may get you to first page load faster. But if a user comes back and you've shipped two small fixes, all those extra wins you get from compressing a bunch files at once fly out the window & you're deep in the red. If you have users that return to your site, and your site is actively developed, bundling is probably a bad tradeoff.

We see similar fixedness in the field all over the place: people freaking love small Docker image sizes & will spend forever making it smaller. But my gosh the number of engineers I've seen fixate on total download size for an image, & ignore everything else, is vast. Same story, but server side: my interest is in the download size for what v1.0.1 of the Docker container looks like once we already have v1.0.0 already shipped. Once we start to consider what the ongoing experience is, rather than just the first time easy-to-judge metric, the pictures all look very different.

Then there's the other thing. The performance reasons for bundling are being eaten away. Preload & Early Hints are both here today & both offer really good tools to greatly streamline asset loading & claw back a lot of turf, and work hand-in-glove with import-maps. The remaining thing everyone points out is that a large bundle compresses better (but again at the cost of making incremental updates bad). The spec is in progress, but compression-dictionary-transport could potentially obliterate that advantage, either make it a non-factor, or perhaps even a disadvantage for large bundles (as one could use a set of dictionaries & go discover which of your handful of dictionaries best compress the code). These dictionaries would again be first-load hit, but could then be used again and again by users, to great effect again for incremental changes. https://github.com/WICG/compression-dictionary-transport

Bundles are such an ugly stain on the web, such an awful hack that betrays the web's better resourceful nature. Thankfully we're finally making real strides against this opaque awful blob we've foisted upon this world. And we can start to undo not just the ugliness, but the terrible performance pains we've created by bundling so much togther.