snowflake VS nanoid

There are so many of them in big data, e.g. Kafka, Spark, Flink, Delta, Snowplow, Finagle, Deequ, CMAK, OpenWhisk, Snowflake, TheHive, TVM-VTA, etc.

Twitter built Snowflake https://github.com/twitter-archive/snowflake/tree/snowflake-2010

I get similar connection-security messages when attempting to connect to the package archive at https://elpa.nongnu.org/ —again, in both *Package* and in *eww*—while elpa.gnu.org seems to be just flat out nonresponsive. However, other https sites including https://duckduckgo.com/about and https://twitter.com work just fine.

Twitter started out with a huge focus on their API. Developers could get almost any data from Twitter they wanted - trends, hashtags, user stats - and they built some really cool stuff with it. This massive amount of open data and the tools people built actually attracted more users to Twitter. Companies could easily hook into the Twitter API to let users share their content on Twitter without leaving their site, and Twitter in turn got even more content on the platform.

Developed and Maintained By – Twitter

PS C:\Users\Admin\Downloads> .\yt-dlp.exe "https://twitter.com/i/broadcasts/1dRKZlzzbMbJB" --verbose [debug] Command-line config: ['https://twitter.com/i/broadcasts/1dRKZlzzbMbJB', '--verbose'] [debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252 [debug] yt-dlp version 2021.09.25 (exe) [debug] Python version 3.8.10 (CPython 64bit) - Windows-10-10.0.19043-SP0 [debug] exe versions: none [debug] Optional libraries: Crypto, mutagen, sqlite, websockets [debug] Proxy map: {} [debug] [twitter:broadcast] Extracting URL: https://twitter.com/i/broadcasts/1dRKZlzzbMbJB [twitter:broadcast] 1dRKZlzzbMbJB: Downloading guest token [twitter:broadcast] 1dRKZlzzbMbJB: Downloading JSON metadata [twitter:broadcast] 28_1445749360700977153: Downloading JSON metadata [twitter:broadcast] 1dRKZlzzbMbJB: Downloading m3u8 information [debug] Default format spec: best/bestvideo+bestaudio [info] 1dRKZlzzbMbJB: Downloading 1 format(s): replay-2750 [debug] Invoking downloader on "https://prod-fastly-us-east-1.video.pscp.tv/Transcoding/v1/hls/jN6CaxrA9lpyuGd0L_kWYPCorYH27HnjKV70nxcVASsjFnGajs0gDWIsABSOYQJd-E7xK7kFJv34xb38IZi6cA/transcode/us-east-1/periscope-replay-direct-prod-us-east-1-public/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCIsInZlcnNpb24iOiIyIn0.eyJFbmNvZGVyU2V0dGluZyI6ImVuY29kZXJfc2V0dGluZ183MjBwMzBfMTAiLCJIZWlnaHQiOjcyMCwiS2JwcyI6Mjc1MCwiV2lkdGgiOjEyODB9.ldktM4fCFRfkP4ZEBfZPKtlAUNAcTPkoz994YJAzWpE/tw_playlist_16813172078088237633.m3u8?type=replay" [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 2409 [download] Destination: Drone Racing League - 2021-22 DRL World Championship _ Twin Cities [1dRKZlzzbMbJB].mp4 [download] Drone Racing League - 2021-22 DRL World Championship _ Twin Cities [1dRKZlzzbMbJB].mp4.part-Frag1 has already been downloaded [download] 0.0% of ~1.09GiB at Unknown speed ETA Unknown ETAERROR: unable to download video data: HTTP Error 400: Bad Request Traceback (most recent call last): File "yt_dlp\YoutubeDL.py", line 2829, in process_info File "yt_dlp\YoutubeDL.py", line 2489, in dl File "yt_dlp\downloader\common.py", line 408, in download File "yt_dlp\downloader\hls.py", line 344, in real_download File "yt_dlp\downloader\fragment.py", line 478, in download_and_append_fragments File "yt_dlp\downloader\fragment.py", line 352, in decrypt_fragment File "yt_dlp\downloader\fragment.py", line 344, in _get_key File "yt_dlp\YoutubeDL.py", line 3256, in urlopen File "urllib\request.py", line 531, in open File "urllib\request.py", line 640, in http_response File "urllib\request.py", line 569, in error File "urllib\request.py", line 502, in _call_chain File "urllib\request.py", line 649, in http_error_default urllib.error.HTTPError: HTTP Error 400: Bad Request

a[href^="https://twitter.com/"] is not part of #yle-consent-sdk-container children

yle.fi###yle-consent-sdk-container:style(visibility: hidden) yle.fi###yle-consent-sdk-container:has(~ .yle__app a[href^="https://twitter.com/"]):style(visibility: visible !important)

If you run that and then look in your Network tab again, you'll see that the response includes a header that says Access-Control-Allow-Origin: *. The * means that this image is allowed to be requested from any origin whatsoever- this is presumably because the NYT want this image to show up in social media shares. This could also be restricted- for example, if for some reason they only wanted the image to be allowed to be requested from Twitter, they could send a header that said Access-Control-Allow-Origin: https://twitter.com.

// ==UserScript== // @name Twitter to nitter // @namespace null // @version 0.1 // @description Redirect twitter to nitter // @match https://twitter.com/* // @run-at document-start // ==/UserScript== redirectToPage("https://twitter.com", "https://nitter.net"); function redirectToPage(page1, page2) { if(window.location.href.indexOf(page1) != -1) { window.location.href = page2 + window.location.pathname; } }

bin: Sometimes it may be desired or even crucial for a package to provide a command line interface, take a testing framework like jest as an example. NPM allows packages to publish executable binaries for this purpose, and as a convention they're usually placed in this directory. Example from nanoid.

The development began two months ago and we have not encountered any scaling issues yet, as the majority of users self-host. Therefore, our priority was focused on building apps. Is there anything you believe we should consider?

You are correct, repeating the same index twice is a mistake. That's why I love open source.

We are using nano id (https://www.npmjs.com/package/nanoid) for all entities, It's stored as varchar in the database.

In middleware.ts I can generate a cookie very easily - doing a check to see if the correlationId cookie is set - if it is, use it, if not, use something like nanoid to generated a unique ID.

Every time /shorten-url is called, nanoid generates a URL-friendly key. Then, the key and a long URL are stored in the mapping for URL redirection. And, the key is stored in a SET object to check if the generated key is duplicated. If the key is duplicated, the error will be thrown.

Just look out for this issue:

https://github.com/ai/nanoid/issues/365

The seed for an LFSR is just the starting value for its cycle; it can be any value in the cycle, and it's public anyway in that you can think of each output of an LFSR as the "seed" for the next output.

But in the case of the "taps", yes, you would want to treat those as a secret if you didn't want someone to predict the next value in the cycle. An LFSR by itself might not be a complete solution for IDs for public consumption, but it does neatly solve the need for pseudo-random non-colliding IDs. Whereas NanoIDs are able to generate values with a ~1% probability of collision over some time period, LFSRs can generate values with a 0% probability of collision over their entire bit space. (And they're composable!)

It looks like NanoIDs are entirely random values reduced to a specific alphabet (https://github.com/ai/nanoid/blob/main/index.js) -- which is approximately similar to:

    tr -dc _A-Z-a-z-0-9 < /dev/urandom | tr -d '/+oO0lLiI1\n\r' | head -c $num_chars

We should consider the following properties when evaluating ID formats and generation algorithms:

1. Private: you shouldn’t be able to gain information about the system using the IDs from an ID alone. E.g. document enumeration attacks like what happened with Parker (https://www.wired.com/story/parler-hack-data-public-posts-im...)

2. B-tree/cache friendly: newly created IDs should all exist in a narrow range of values. This is helpful for databases.

3. Stateless: ideally you shouldn’t need to know the current state of the system to create a new ID.

4. Human-friendly: IDs should be easily dictated, copied, pasted, etc. This means they should be encodable as text that is short and does not include ambiguous characters.

Some of the these properties are in conflict. Statelessness is achieved by randomly generating long IDs, but people don’t like reading or typing long IDs.

Different use cases will need these properties in varying amounts. If you don’t intend to expose the IDs to users, (4) doesn’t matter. Just use long, randomly generated byte strings prepended with the date. Most databases have a UUID type that fits the bill.

If users are going to be working with IDs, that’s more complicated. If not every document has a user-facing ID, just go with the non-user-facing ID like before, and generate a shorter, random, stateful ID as needed.

I don’t think NanoID prepends the date, which means it won’t be efficient when inserting large numbers of IDs into a large index. They also default to using ambiguous characters like 1 and I and l. Also no error code. But they are shorter than UUIDs. So it doesn’t meet property (2), and it only kind of meets property (4). NanoIDs are random, so you’re probably safe from enumeration attacks (1). NanoIDs mostly leave Statelessness as a decision for the user. They have a nice tool that helps estimate how long the IDs should be (https://zelark.github.io/nano-id-cc/) for a given collision resistance.

I think we can do better overall. Bitcoin uses a good encoding scheme called base58check. It generates fairly short IDs and uses an error code at the end. I think it could be refined for non-bitcoin purposes, but it’s already pretty good.

A 128-bit value like the ASCII string “hackernewstestid” is encoded as “Dtajqjz5pptWcmGrNcwBx7”. It’s about 2/3 the size of the equivalent UUID, even with the (unnecessarily long for this use case) error checking code.

I’d like to see a small ID standard that meets the above requirements and has a choice for either stateless and long or stateful and short. Maybe another choice for secure random or insecure. But all options would have binary form and a text form. The text form would use something similar to base58check, but probably with a smaller (or user-determined) length for the error code.

Rio Docs Zod Github Nanoid Github Sharp Github

I really like https://github.com/ai/nanoid which is available for a wide range of programming languages. It creates short, random, and URL-safe IDs.