cargo-n64 VS libdragon

Nintendo 64: MIPS R4300i, supported by upstream LLVM and GCC, some Rust support

I’m not sure the N64 would be easier. The graphics hardware in the N64 is complicated.

In some ways the console is really modern (perspective-correct texture interpolation, z-buffering, subpixel positioning). On paper, it seems a lot like a more modern system. But the hardware complexity is a beast. and the design is permeated with bottlenecks. Half of the graphics pipeline is taken up by the RSP, which is a fully programmable vector processor--sounds cool, but in practice it is unapproachable. 4 KiB of texture memory (and if you use a palette, half of it is the palette). RAM bandwidth is barely fast enough to handle 3D scenes.

https://www.copetti.org/writings/consoles/nintendo-64/

The folks working on LibDragon have done a lot of work to make homebrew a better experience, but you’ll note that support for 3D graphics is still only in a preview branch.

https://github.com/DragonMinded/libdragon

Per the Portal 64 creator, the library in question was libultra,[1] which is owned and was distributed exclusively by Nintendo but has been extensively pirated.

There are open source alternatives, including libdragon[2] and libn64[3]. Not sure how feasible replacing libultra with either of those would be considering the scale of the project.

1: https://n64brew.dev/wiki/Libultra

2: https://libdragon.dev/, https://github.com/DragonMinded/libdragon

3: https://github.com/mikeryan/n64dev/

To do 2D on N64, I suggest you to use the libdragon unstable branch https://github.com/DragonMinded/libdragon/wiki/Unstable-branch

I was thinking of doing this for Wii (via GRRLIB), or potentially N64 (via libdragon), but Godot would probably be too resource heavy for the N64, so I'm leaning more towards the Wii.

And an open source sdk at https://github.com/DragonMinded/libdragon libdragon is well on its way to surpassing Nintendo's libultra, but the discord has people knowledgeable about both.

It is pretty much understood in most aspects that pertain regular software development, though there are still corners that are investigated.

The most accurate and fast emulator right now is Ares (https://ares-emu.net), which bundles the Vulkan-accelerated RDP emulation with a recompiler for both CPU and RSP. It is extremely accurate in many regards and in general much closer to the real hardware than any other emulators (with cen64 being a close second). Other emulators manage to run most of the game library but using several hacks, while Ares keeps a zero-hack approach, so not everything works, but it is for instance far more compatible with advanced home-brew stuff which use the hardware in ways that the Nintendo SDK did not.

The most advanced open source library for N64 development is libdragon (https://github.com/DragonMinded/libdragon) which is currently growing very advanced RSP ucodes that do things that are not possible with Nintendo SDK. For instance, it was recently merged a command list support to send commands from CPU to RSP without any lock in the happy path, and fully concurrent access from both the processors. Another example would be its DMA support for fetching data from ROM that exploits undocumented partially-broken features of the RCP that were previously unknown to allow for misaligned memory transfers.

The most accurate source of hardware documentation is the n64brew wiki, which is slowly gathering accurate, hardware-tested information on how the whole console works. https://n64brew.dev/wiki/Main_Page. Unfortunately, it's still lacking in many areas (eg: RSP). It's a painstaking long work because there are many many documents floating around with partial or completely wrong information.

Well...it would be the engine that you write for it, likely using libdragon or something similar as the SDK.