This guide focuses on concepts that stay stable as software changes: network topology, IP addressing, server and client roles, and the difference between small API requests and large model files. Specific tools appear only as implementation examples.
The goal is a setup where the RTX PC can use the AI node as reliably as a local application.
The 30-Second Answer
Wi-Fi works, but a cable gives stable latency and makes troubleshooting simpler. Treat Wi-Fi as a fallback.
Connecting both machines to your router or a switch is the simplest topology. A direct cable also works; static addresses make it the most predictable to manage.
The AI node runs a model server that listens on the LAN. Applications on the RTX PC send requests to its address and port.
API requests are small. Link speed mainly affects how fast you can move model files and large outputs between machines.
Why Connect the Two Machines
The purpose of the connection is compute separation, not file sharing. The language model stays loaded in the AI node’s memory, and the RTX PC uses it over the network without spending its own VRAM.
- The RTX GPU stays free for image generation, rendering and real-time work.
- Large models remain resident on the node instead of being reloaded for every session.
- One node can serve several clients, such as a desktop, a laptop and scripts running elsewhere.
Whether a second machine is worth it at all is covered in RTX PC + Unified Memory AI Node.
Two Kinds of Traffic: API Requests and Model Files
Most confusion about network speed disappears once you separate the two kinds of traffic:
| Traffic | Size | How often | Sensitive to |
|---|---|---|---|
| API requests and responses (prompts, generated text) | Small | Constantly during use | Latency and reliability, rarely bandwidth |
| Model files (weights) | Very large | Occasionally, when adding or updating models | Bandwidth |
| Images, video frames and project assets | Medium to large | Depends on the workflow | Bandwidth |
The practical rule
For text generation, the speed of the model on the node matters far more than the speed of the network. Bandwidth becomes noticeable when you copy models between machines or move large batches of images and frames.
Three Topologies: Direct Cable, Router, Switch
| Topology | How it works | Strengths | Watch for |
|---|---|---|---|
| Through your router | Both machines plug into the home or office router | Simplest; addresses are assigned automatically; both keep internet access | Router ports may be limited to 1GbE |
| Through a switch | Both machines plug into a switch connected to the router | Allows faster links (2.5GbE or 10GbE) even if the router is slower; easy to add more devices | The switch, both network ports and the cables must all support the target speed |
| Direct cable | One cable runs between the two machines | No extra hardware; a private, dedicated link | Static IP addresses are the most predictable setup; internet access needs a second connection such as Wi-Fi or another port |
Modern network ports detect the cable type automatically, so an ordinary Ethernet cable works for a direct connection. A direct link can use self-assigned addressing on some operating systems, but static addresses make the connection easier to manage and troubleshoot. If you only need the two machines to talk to each other, a direct cable is fine. If other devices should reach the node, use the router or a switch.
IP Address Basics
Both machines need addresses on the same subnet. The approach depends on the topology:
| Topology | Recommended addressing | Example |
|---|---|---|
| Router or switch | Keep automatic addressing, but reserve a fixed address for the AI node in the router’s settings | Node always receives 192.168.1.50 |
| Direct cable | Assign static addresses on both machines in a private range different from your main network | Node 10.10.10.1, RTX PC 10.10.10.2, subnet mask 255.255.255.0 |
- A fixed address for the node means client settings never break after a restart.
- Avoid addresses inside your router’s automatic range to prevent conflicts.
- Local hostnames can replace IP addresses where your network supports them, but a fixed IP is the most predictable option.
Server and Client Roles
Runs a model server that listens on a network port. It must be configured to listen on the LAN interface, not only on the local loopback address, and the firewall must allow that port on the private network.
Applications, scripts and editors send requests to the node’s address and port. To the client, the node behaves like a web service on the local network.
Many local model servers offer an HTTP API that follows a widely used chat-completion format. When client applications speak that format, you can change the server software or the model on the node without changing the client configuration beyond the address.
Choosing Link Speed: 1GbE, 2.5GbE or 10GbE
| Link speed | Fits | Requirements |
|---|---|---|
| 1GbE | Everyday API use; occasional model transfers where waiting is acceptable | Available on almost every PC and router |
| 2.5GbE | Regular model updates and moderate image or asset transfers | Both ports and the switch or router must support 2.5GbE; common Cat5e or Cat6 cables usually suffice for typical home distances |
| 10GbE | Frequent large model transfers, shared model storage, heavy video or frame workflows | Both ports and a 10GbE switch, or a direct cable; Cat6A recommended for longer runs; higher power and heat on some adapters |
Start from the slowest link in the chain. A 10GbE port on one machine does not help if the other machine, the switch or the cable is limited to 1GbE.
Where Model Files Should Live
| Location | Strengths | Trade-off |
|---|---|---|
| Local SSD on the AI node | Fastest model loading; no dependence on the network during use | Storage must be planned on the node; copying new models takes time once |
| Shared folder on the RTX PC | One copy of each model | Model loading depends on network speed and on the RTX PC being on |
| NAS | Central storage for several machines and backups | Loading speed limited by the network and the NAS; another device to maintain |
For most two-PC setups, keep active models on the node’s local SSD and use shared storage or a NAS for archives and backups.
Implementation Example: Serving a Local Model on the LAN
The pattern is the same regardless of the software you choose:
- Install a model server on the AI node and confirm it works locally.
- Configure it to listen on the LAN interface and choose a port.
- Allow that port through the node’s firewall for the private network only.
- From the RTX PC, confirm the node responds at its address and port.
- Point client applications at that address instead of a local server.
Ollama, LM Studio and the llama.cpp server are common examples of software that follows this pattern. Each documents how to enable network access, and their options change between releases. Follow the current official documentation linked in the references rather than fixed commands copied from articles.
Keep It on the LAN
- Many local model servers do not require authentication by default. Treat the node as a private service.
- Do not forward the server port from your router to the internet.
- Restrict firewall rules to the private network profile.
- If you need access from outside your home or office, use a VPN or another secure remote-access method rather than exposing the port.
Troubleshooting Checklist
| Symptom | Likely cause | Check |
|---|---|---|
| The node cannot be reached at all | Different subnets or a cable issue | Both addresses share the same subnet; link lights are on |
| The node answers ping but not the API | Server listening only on localhost, or firewall blocking the port | Server listen address and firewall rule |
| Connection works, then fails after a restart | The node received a new address | Address reservation or static IP on the node |
| Transfers are slower than expected | One link in the chain runs at a lower speed | Negotiated speed on both ports, switch ports and cable rating |
| Responses are slow but the network is idle | The model is too large or slow for the node | Model size and quantization on the node, not the network |
Frequently Asked Questions
Can I connect two PCs directly without a router?
Yes. A single Ethernet cable between the two machines works, as modern ports detect the cable type automatically. Some operating systems can self-assign addresses on a direct link, but static IP addresses on both sides are easier to manage and troubleshoot. Each machine needs another connection if it also requires internet access.
Is Wi-Fi good enough for a two-PC AI setup?
It can work for light use because API requests are small, but wired Ethernet provides more stable latency and much faster model transfers. Use a cable for the AI node whenever possible.
Do I need 10GbE to use a local LLM from another PC?
No. Everyday prompts and responses run comfortably over gigabit Ethernet. 10GbE helps when you move large model files often, share model storage, or transfer large images and video frames between machines.
Should model files be stored on a NAS?
A NAS is useful for archives and backups, but active models load fastest from the AI node’s local SSD. Most setups keep working models on the node and use a NAS or shared folder as secondary storage.
Is it safe to open my AI node to other devices on the network?
It is reasonable on a trusted private network if firewall rules are limited to that network. Many model servers have no authentication by default, so never expose the server port directly to the internet.
Official Technical References
Technical references only. These links support the specifications and concepts on this page and are not purchase links.
Next Steps
Revisit what each machine should run, or return to the Local AI guide: