Question about A.I.

[tascoast]

It's difficult to imagine where it ends with AI Co-pilots embedded into Visual Studio, VSCode and other IDE environments, that then are gobbled up by the big players and integrated into the development ecosystem.

A key issue with AI, besides providing useful algorithms for modern hearing aids, for example, is that any serious AI requires use of finite power supplies and hardware, Taiwan being a crucial hub, high-end processors currently denied to those like China, who can still model with existing arrays, but the race is on to control the processing power and manufacture and application.

[Nokkaelaein]

If the AI, hardware or software, could be contained locally I could probably live with it. [...] However, if it's connectivity to the cloud is required (which it always is)

No, it isn't. For example, I currently only use local AI, as do many of my developer friends. At the moment my go-to daily driver is a recent 30 billion parameter model that I currently run in a local Ollama server (https://github.com/ollama/ollama), that in turn using the llama.cpp backend (https://github.com/ggml-org/llama.cpp). This performs quite remarkably for the types of tasks I use it most (an extended VSCodium providing local editor integrated programming tools). I use quantized models in gguf format, and that coding related use is the most frequent task I have for these. Another use I have is actually developing software that utilizes local solutions like that, local quantized models and APIs, for things like (100% local!) data analysis in enterprise use.

I've also made my own local pipelines for using GLiNER models for entity recognition and do various sorts of categorization / textual entity separations using those (https://github.com/urchade/GLiNER)

For things like translation, I tend to use the text-generation-webui project which runs as a local web server and can be interacted with in your browser, again 100% locally (https://github.com/oobabooga/text-generation-webui). If I'm translating a language from some other source than a readily digital text, and the text contains glyphs that aren't familiar to me, written in character systems I don't personally know, I can take a photo of said text / scan the said text, and first run it through OCR to convert that to digital text, and then I can continue in an established translation session. I also have AI transcription tools in place that I can use to transcribe speech to text, so if I'm dealing with an audio recording of a language I don't know, I can again first produce actual text from it, then translate it from there. In other words I have 100% local means to AI translate from text, photos and spoken word, when needed. I've integrated these tools into my private Linux environment based on MX, and they can all be used without any internet connection present, and deployed on a new computer in one go with my work configurations ready in place.

I never use AI to produce any of the text I write here, in any way. If I did, I'd attach a disclaimer.

Anyway: running local, open - and indeed freely available without strings attached - tools that you install once and which then function from there on as you wanted and needed, without requiring any internet connection even intermittently... this is where it's at, imo. No matter whether we're talking about AI or, say, music or graphics production, heh.