I tried using `tailscale funnel` against a dummy server `python -m http.server`, and within 10 seconds the bots started to check for vulnerabilities.
Tailscale warns you about how enabling it will issue an HTTPS certificate which will be in a public ledger. But I wasn't expecting it to be this quick.
I use Headscale, an open source implementation of Tailscale control server. And it doesn't have funnel functionality implemented out of the box, but I use a custom Traefik proxy manager Web UI in which I can expose ports on different Tailnet nodes.
In order to avoid exposing something unnecessarily in the certificate transparency logs, I use a single wildcard certificate, so all the subdomains are not listed anywhere automatically.
I use the same approach for services hosted in the internal subdomain, because I don't want everyone to know what exactly I'm running in my homelab.
Another approach I’ve seen is to route public access from Traefik/nginx through a single Cloudflare tunnel instead, and Tailscale/Headscale can be left for private network and server access.
The traefik box can have the single Cloudflare tunnel , and tailscsle can hang out behind the scenes.
This way tailscale funnel doesn’t need to be public.
There is the self hosted Cloudflare alternative that’s escaping my mind right now too.
Cloudflare also issues certs and logs them in transparency logs. If you do not create a wildcard cert in Cloudflare, your subdomains will leak. And Cloudflare offers free wildcard certs only on the domain root.
You don't have to serve any certificates on the default website. Web server would just fail TLS connection, since it doesn't have a certificate for it.
Not sure if this applies to all web servers, but at least Caddy and a few others support this.
A DoS that will disappear once you close the funnel. Tailscale are proxying the traffic so your public IP isn’t exposed. Your choice of port makes no difference.
This may discover services, but not hostnames. If the server does not disclose them (e.g. in the certificate used on the IP host), an attacker doesn't have much further to go on.
All the dev servers I've used over the past 10 years come with warnings that they're not security hardened, so I'd be wary of using `tailscale funnel` even though it is awesome to share like that so easily.
I wish this trend of “security through obscurity” should mean that all info should just be exposed would die, its silly and lacks basis in reality.
Even within infosec, certain types of information disclosure are considered security problems. Leaking signed up user information or even inodes on the drives can lead to PCI-DSS failures.
Why is broadcasting your records treated differently? Because people would find the information eventually if they scanned the whole internet? Even then they might not due to SNI; so this is actually giving critical information necessary for an attack to attackers.
The issue is not that obscurity per se is bad, but relying _only_ on obscurity is absolute the same as not having any security measures at all.
With the public ledger or not, you will still need to implement proper security measures. So it shouldn't matter if your address is public or not, in fact making it public raises the awareness for the problem. That's the argument.
Until it gets obscure enough that we start calling it “public-key cryptography”. Guess the prime number I'm thinking of between 0 and 2↑4096 and win a fabulous prize!
If you replace "security by obscurity" with "Kerckhoffs's principle", yes, absolutely!
The problem with using regular everyday obscurity is that it usually has a small state space and makes for terrible security, but people will treat it like it is cleverly hidden and safe from attackers
If I guess the IPv4 you're thinking of between 0 and 2↑32, ready or not, you win a free port scan
> So it shouldn't matter if your address is public or not, in fact making it public raises the awareness for the problem. That's the argument.
Forget about the internet, we've had almost 100 years to prove we can secure identity theft. And the best thing we can do is to keep our SSN's secret -- security through obscurity. Keeping your SSN private reduces your personal attack surface.
We've had 50 years to secure the internet, and yet, we still have zero day attacks. Nuclear submarines try their best to keep their locations a secret? Why? You cannot attack something you cannot see or hear.
Battleship sounds like a good analogy, but is very different because you don't have other options to "secure your ship" besides obscurity. If you had other options, let's say a sonar or moving your ship, they would definitely be used along with obscurity.
Besides, the time to scan the whole board is too time consuming in a battleship game, but scanning the whole internet on the other hand only take a few minutes[1]
You're talking IPv4 here, not IPv6. A 24 bit network has 254 addresses in IPv4. A 64bit subnet in IPv6 has 2^64.
If you can scan 1M ipv6's in a second, you can maybe scan 1 subnet in 584,942 years.
So if you're a firewall, and you notice scanning from a particular ip or network, it's easy enough to block them.
Also if you are scanning IPv4, you're not scanning addresses behind the NAT'd routers -- which is also effectively a form of obfuscation. So I would argue it's not the entire internet.
Okay, but we're not talking about that here. This is very much the case of a service being exposed that shouldn't be and relying on obscurity to try and avoid actually getting compromised
If something was temporary then it’s likely that it wouldn’t have been found in a meaningful amount of time to be exploited.
As an only line of defence it’s not good, but its also not good to hand-deliver your entire personal information to fraudsters and then claim that the systems should be more robust.
If you have a target on your own back thanks to cert transparency logs, it's a bit like closing the barn door late for you to find fault in your own being in Texas when sharpshooters are about. If your only defense was obscurity, your ass is hanging out, and it's no one's fault but your own when you find fault with others for simply saying so.
IME, moving ssh off the standard port reduces bot scanning traffic by >99%. Not only it means less noise in the logs (and thus higher SNR), but also lowers the chance you're hit by spray-and-pray in case there's a zero day in sshd (or any other daemon really).
True, but I hardly open any ssh to the wide world. I would only allow it inside a closed network anyways. HTTP on the other hand _needs_ to be exposed on 80 or 443 (not technically, but in practice)
The context of the conversation is that the address becomes publicly visible so you get hit with port scanners and script kiddies looking for vulns. Moving off standard ports does help but many of those are also going to look at ports like 2222 or 8022 and treat them as ssh.
It's not hard to just send something like `nmap -sV -p- <ADDRESS>` (or better, use like rustscan.) and you'll discover those ports and the services.
On the other hand, just install something like knocked and you don't have to do much. Knocking is not a difficult thing to set up.
Which is something that makes a notable difference. It’s telling the bots the OP listed are trying Vite endpoints, they’re targeting folks doing short term local web development. Removing obscurity and indicating relative likelihood of still being online is a big shift.
Yes. Yes, of course they do. Check for example https://crt.sh with your domain name to see the glorious public history of everything the certificates tell about your domain.
I don't see why people don't just run their own CAs more for private stuff.
If exposed for others I think the wildcard cert is also what I did, but most tutorials have you issuing certs via ACME for internal or local-only things which doesn't even need to happen.
I personally run my own CA and even setup an ACME server and internal DNS. Nobody knows what I am doing there.
It was common to set up your own CA at one point, especially when DNS management was more manual, However it presented a huge attack surface and was challenging to manage.
A compromised private CA can lead to widespread breaches, affecting various systems and applications that rely on its certificates.
The CAB forum working groups being explicitly prohibited from working on private networks (at least historically) and market incentives also produced a situation where you can't really reduce the blast radius.
ECS1 attacks on AD CS is probably the best publicly documented case for further research.
The happy path is often manageable, but still complex, bland any mistake will result in huge risks.
For me, the value proposition isn’t there. I can get a wildcard domain signed from let’s encrypt and it works out of the box on every device, and you don’t have to deal with the fact that some/many appps will ignore your OS certificate rules.
I use a similar setup, but for anyone following this guide i would not recommend hosting your custom oidc server behind the same tailnet it authorizes.
Any configuration issues will lock you out entirely and you will need to have tailscale support re-enable an oauth provider and its not reversible.
I use an oauth provider to log in to tailscale and keycloak internally as an oidc provider for service to service auth.
I don't know enough about networking as I should, so to plug for my gap in knowledge, I generally prefer to use more comprehensible (to me) forms of security. And a feature like this:
> Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine.
kinda worries me (given also IP spoofing is possible?), compared to SSH keys whose mechanism is more obvious and thus easier to trust.
I definitely like the idea of Tailscale as an extra layer of protection, but I'm not sure I'd loosen existing protections while using it, whereas many Tailscale articles often present it as a panacea for internal-network-over-the-internet security. Are my concerns misplaced?
> kinda worries me (given also IP spoofing is possible?),
It's not, Tailscale authenticates incoming connections. (Note that we're not talking a regular SSH connection to the server's public IP here. You'd connect to the server's SSH daemon through Tailscale.)
> It’s a subscription product, but it has an insanely generous free tier that covers basically anything you’d ever want to do as an individual.
Tailscale do have a very nice product, but privacy-conscious users should be aware that you must disable Tailscale's real-time remote collection of your behavior on your “private” network. See KB1011: https://tailscale.com/kb/1011/log-mesh-traffic
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.io). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326
“When you use the Tailscale Solution, we collect limited metadata regarding your device used to access the Tailscale Solution, such as: the device name; relevant operating system type; host name; IP address; cryptographic public key; user agent (where applicable); language settings; date and time of access to the Tailscale Solution; logs describing connections and containing statistics about data sent to and from other devices (“Inter-Node Traffic Logs”); and version of the Tailscale Solution installed.” (emphasis mine)
Anyway, the reason I quoted that part of your post is because Tailscale are using some Fear, Uncertainty, and Doubt tactics here by naming the privacy-preserving option “no-support”, and if you are a free user then you aren't getting support from them anyway, so there should be no downside to keeping your private network private :)
Tailscale is a secure connectivity tool that puts the highest value on the privacy of your packets. But we made an intentional choice from day one that we weren't going to try to be an anonymity tool. Quite the opposite in fact! We're an identity-centric network.
Anonymity tools, like Tor, need to be architected very differently. They trade away speed to reduce traceability. They are hard to inspect and diagnose and debug, as a feature. They make enemies, both political and corporate. They are inherently hard to audit and control, by design. In short, they are the exact opposite of what you want your corporate (or even homelab) network to be.
We believe anonymity tools are essential to safe network infrastructure and a free society. But, those tools are made by other people.
…
But if you’re looking for complete anonymity online, Tailscale is not the tool for you. Y'all, we're an identity-centric network with a centralized control plane. You should assume law enforcement can easily find out that you use Tailscale. Tailscale packets are pretty easy to detect, so you can assume they could know, through ISP logs, the shape and size of data you send between different nodes in different places (albeit without knowing the decrypted packet contents). You should assume they can correlate that flow metadata with your login identity.
Open and Close events are not related to identity or anonymity, so that post isn't in itself relevant. It does show that the team is very pragmatic, though.
I get why they capture this data, and by doing so they managed to build an exceptionally great service. But I also understand why one would be uncomfortable with exposing this data.
This isn't relevant to what you were replying to. Parent comment is complaining that there are logs being sent out about what is happening on his private network, he's not expecting anonymity on the internet like Tor (which is what your link describes).
The issue parent is complaining about is that tailscale sends data about communication data in your _private_ network to their servers. The reply to that is a blog post that clarifies that tailscale is not Tor. Irrelevant, nobody is comparing it to Tor.
That section of the policy simply describes how the system works. It's very valuable information for enterprise customers who are effectively their entire market revenue-wise. Think access logs, intrusion detection, and so on. I do not interpret their policies such that they are processing the information you added emphasis to beyond what is necessary to serve the customer. What evidence do you have to the contrary?
The irony of your post, which brings up Fear Uncertainty and Doubt, is certainly not lost on me. I'm also sure you could just ask apenwarr directly for clarification.
> I do not interpret their policies such that they are processing the information you added emphasis to beyond what is necessary to serve the customer. What evidence do you have to the contrary?
Respectfully, you are failing to appreciate the full scope of the problem. It doesn't matter what Tailscale do with the data. The log contents don't matter at all, only the fact that a network connection was made. Every network connection you make creates metadata about you, and the Internet itself — the path between me and Tailscale's logging endpoint — is always listening.
Think what conclusions can be drawn about a person's behavior from a log of their network connections. Encryption doesn't matter, because we're just talking about metadata; each connection's timestamp, source, destination, and port. Think about the way each additional thing-which-makes-network-requests increases the surveillance value of all the others.
Straight away, many people's NTP client tells the network what OS they use: `time.windows.com`? Probably a Windows user. `time.apple.com`? Probably Mac or iOS. `time.google.com`? You get the idea. Yeah, anyone can configure an NTP client to use any of those hosts, but the vast vast majority of people are taking the default and probably don't even know what NTP is.
Add a metadata point: somebody makes a connection to one of the well-known Wi-Fi captive portal detection hosts around 4PM on a weekday? Maybe somebody just got home from school. Captive portal detection at 6PM on a weekday? Maybe somebody just got home from work. Your machines are all doing this any time they reconnect to a saved Wi-Fi network: https://en.wikipedia.org/wiki/Captive_portal#Detection
Add a metadata point: somebody makes a network connection to their OS's default weather-widget API right after the captive-portal test, and then another weather-API connection exactly $(DEFAULT_INTERVAL} minutes later? That person who got home is probably still home.
Anyway, you get the point that this stuff adds up! The problem with Tailscale is that its default behavior exposes metadata about entire additional classes of traffic in addition to all the examples I just mentioned that my devices were already leaking. Tailscale would have me start telling the Internet “hey I'm here and doin' stuff!” every time I read or write any file on my NAS, every time I use Steam Link remote play over LAN, every time I SSH or RDP into any of my other machines.
My behavior would be exposed to every layer of service provider along the way: my ISP, my ISP's ISPs, the cloud provider Tailscale use to host their surveillance endpoint, Tailscale themselves if they so choose, whatever creepy secret spy implants we're not allowed to know about. No thanks! If you want to be private, you must be silent.
> My behavior would be exposed to every layer of service provider along the way: my ISP, my ISP's ISPs, the cloud provider Tailscale use to host their surveillance endpoint,
Maybe I'm missing something here but I'd guess that data is encrypted and not a free for all of open data that any old ISP could snoop on. If not that'd be a serious issue.
Not to say that you don't have some good points. Even just the pattern and timings of that data being sent could be exploited. Also TS would still have that full data.
Though I'd have to study the details. Do they aggregate and then send it at regular intervals, etc? In the end would it be that much worse than what Apple, Google, Microsoft collect?
> Maybe I'm missing something here but I'd guess that data is encrypted and not a free for all of open data that any old ISP could snoop on.
Yes, you are missing the entire point. You are talking about data. I am talking about metadata — data about data. The contents of each log request are a total red herring. Just pretend that the encrypted log messages are a single bit, just a way to increase a counter that “something has happened” on a person's Tailnet.
The encrypted log message structure does tell Tailscale “this particular machine on the Tailnet talked to this other particular machine on the Tailnet at this time”, and one should assume Tailscale decrypt and interpret those details, but what I'm talking about is the ability for any part of the network path to interpret those log connections without decrypting them as “somebody is using their Tailnet right now in any capacity”, and when, and from where, and the ability to combine that new class of metadata with all the other metadata our modern OSes are constantly generating.
> Do they aggregate and then send it at regular intervals, etc?
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.com). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
“This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
Eh, as a network administrator you want the netlogs on by default and you very clearly onboard everyone to the network with a memorable warning to do their personal browsing over some other interface. You've usually got at least some minimal audit requirement on any network with high value stuff on it.
It's probably not great that someone trying to use the free sample product lands in the same netlogging regime as the work network default, but I suspect thats more about allocation of attention and priority which understandably goes to the companies that make up approximately all of their business. Keeping the free sample product around after its long bern clear "this is for work computers" is just one of those things. The "no support" suffix on a setting is not to me the smoking gun you make it out to be, and I'm pretty hardcore in my attitudes about surveilance.
I agree it's the wrong default for a purely personal user, but TailScale has enough "good faith actor" points with me that I'll give them the benefit of the doubt on malicious/evil dragnet surveilance ambitions. What could they possibly want with the data of a group of people who are by construction not spending money on a VPN? They'd be storing it at a loss.
> What could they possibly want with the data of a group of people who are by construction not spending money on a VPN? They'd be storing it at a loss.
This is the exact point where our conclusions diverge.
Why are they sending themselves so much "useless" data-intensive logs by default, from their non-paying clients that accounts for roughly ~95% of the userbase and from a profitable business perspective, largely ineligible for troubleshooting support? For me, the only logical conclusion is that the data is valuable to them.
As someone who also cares about privacy, hear are a few things that IMO suggest that free customers' logs are a part of their business model:
* Their documentation has plenty of references to security, but no references to privacy outside of the privacy policy.
* They have all but eliminated any revenue stream from average user when they made an unsolicted announcement that they upgraded their free plan to allow 100 devices and 5 users.
* The content they sponsor for marketing/advertising seems targeted for consumers instead of networking professionals. I don't see Cisco and Palo Alto Networks sponsoring every Linux/self-hosting podcast or YouTube channels for example.
* Even the flag-name for turning off logging is mild deterrent based on what you will lose (`--no-support`) as opposed to being neutral '--no-logging' or truly descriptive like most FOSS companies that are not pushing an ulterior motive such as '--no-analytics'.
* logs cannot be disabled for phones
* In my experience, disabling logs was perhaps the only thing that was not configurable through the GUI
I'm into privacy and still relatively new on the networking scene thanks to setting up OpenWrt on my router. Am I correct that when tailscale updates/hijacked resolv.conf, subsequent DNS resolution is passed onto them on visited websites even when tailscale is not being used? No, they can't "read" your traffic, but if I understand things right, they know every website you visited and for how long, which is more than enough data for a rich marketing profile. That was my takeaway before I jumped ship for a self-hosted solution.
My understanding is that they have the holy grail of data because they are getting all of your LAN, WAN and mobile network traffic. I'm not aware of (m)any companies whose business model allows access to all three? It's like if your ISP and your Mobile Network had a baby on your local server, and that baby reports every website you visit.
> Am I correct that when tailscale updates/hijacked resolv.conf, subsequent DNS resolution is passed onto them on visited websites even when tailscale is not being used?
I think you're incorrect in the default settings, even when tailscale is active.
By default, your tailnet's devices use their local DNS settings for all queries. To force clients to always use the nameservers you define, you can enable the Override DNS servers toggle.
> I think you're incorrect in the default settings
What mac-attack is correct about is that by default, `tailscaled` sets itself as the only DNS resolver and proxies all DNS requests to your non-Tailscale nameservers. Citations:
“`100.100.100.100` or Quad100 is a special Tailscale IP address […] that provides essential local services. It operates similarly to the localhost address (`127.0.0.1`) but serves only Tailscale-specific services. These services include a DNS resolver.”
“One of the services provided by Quad100 is a DNS resolver running on port 53 (1100.100.100.100:531). A DNS resolver is a service that translates IP addresses to hostnames like `google.com` or `macbook.tailnetname.ts.net`. Quad100 is a stub resolver, similar to systemd-resolved, except with extra features.”
“The upcoming Tailscale 1.8 release implements all of the above [other DNS managers], which should hopefully make DNS on Linux just work, no matter how your machine is choosing to do it.”
“Tailnets created on or after October 20, 2022 have MagicDNS enabled by default.”
It does say “While Quad100's DNS resolver operates locally without logging, forwarded requests might be logged by configured nameservers.”, but the fact remains that the Tailscale software is very aggressive about taking over all DNS resolution on your system. Once that is done, the option of whether or not `tailscaled` overrides your default nameservers can be configured remotely without you knowing it's happening!
I'm split on this. According to your links, it tries to cooperate with the system resolver. If it can't find a way to do it, then yeah, it kinda has to replace it.
Of course, they could put this much more front and center in the docs, so that if you're running some funky setup and know what you're doing, you should be able to easily do it - which you probably can with the `--disable-dns thing`. But putting it in a prominent spot in the docs could help to not overlook this.
I've just checked the setup on a machine running systemd-networkd and resolved, and resolv.conf wasn't touched. It only added a specific dns setup for the tailscale0 interface, which only covers my tailnet name and ips. It doens't even show as a fallback or whatever in the global section.
> the option of whether or not `tailscaled` overrides your default nameservers can be configured remotely without you knowing it's happening!
I mean, there's two situations. Either we're talking about a "pro" environment, where corp vpns taking over your local network config, as much as I hate it, isn't exactly news. Then there's the personal plans users, in which case, if the DNS changes without you knowing, you probably have way bigger problems.
Logging everyones network data/metadata would likely be illegal under employment law in Norways. Other European countries may have same/similar rules. GDPR may also apply.
So be careful with how broadly you apply that default.
One useful additional aspect to Tailscale that I've not seen mentioned so far, is the integration with Mullvad.
Using that you can get the benefit of their network servers, appearing just as standard Tailscale exit nodes, which is handy if you need to geo-shift traffic at all.
> One useful additional aspect to Tailscale that I've not seen mentioned so far, is the integration with Mullvad.
Indeed, this saved my butt earlier this year when I was at Qatar airport and they tried to MITM most of my connections (including to mullvad.net). Luckily, they didn't MITM tailscale.net, so I could log in, enable Mullvad, and thereby secure my entire traffic.
Indeed, Mullvad is a VPN provider (https://mullvad.net/en) they provide several privacy focused services including a VPN and a browser.
Using Mullvad (or other VPN products) allows someone to make their traffic originate from specific countries, which can allow for testing to see what it'll look like from different countries, or to access content which is restricted to people in a specific country. It can also allow for bypassing age check restrictions that apply to specific geographies (e.g. the UK)
Integrating it with Tailscale makes it easier to use (if you're an existing tailscale user) as instead of having to install and manage a separate product, it integrates with your existing tailscale setup, allowing you to dynamically choose a mullvad exit node in a give country, and then your Internet traffic will appear from that country.
(that all sounds somewhat ad-like but I'm not in any way affiliated with Tailscale or Mullvad apart from being a user (Tailscale did give me a nice hoodie one time for writing a blog though ))
Accidentally wiring everything to everything else sounds kind of scary.
There's 1 or 2 things I wouldn't mind securely exposing to the internet (like Plex) but nothing I need so desperately while I'm out and about that I'd even want to take that risk.
Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine. This is particularly handy when you SSH from a phone, as proper credential management is a bit of a nightmare there.
this has me worried. i would not want that. i use zerotier, not tailscale, but the principle is the same. i have my laptops and my phone connected to my servers. given that all of those machines are already on the internet, connecting them into a virtual network does not add any risk in my opinion. (at least as long as you don't use features like the above). all i get is a known ip address for all my devices, with the ability to connect to them if they have an ssh server running. when i am outside the primary benefit is that i can tell which devices are online.
this is for teams where you don't want to create passwords or keep track of ssh keys for everyone by hand. it greatly simplified our server usage as we can simply ssh user@machine and it just works. you can create access controls for it as well.
You can use it like ngrok, and I'm sure you could configure wireguard and ngrok to give you something similar to what Tailscale does, but Tailscale does it out of the box, with polished and well built client and server apps.
I'm no infra guy, I'm just a former front-end eng, but it gives me the confidence to expose media centres and file servers etc to "the wild" without it being public.
Using Jellyfin to watch content from my home server on my iPad while I'm away from home is as "easy" as Disney or Netflix with Tailscale, just installed the clients and servers and .. voila?
I was an infra guy early in my career, and I'm still savvy, and I still prefer using Tailscale. It's very polished and reliable.
But personally, I'm past the point of wanting to fiddle with things like this and would much prefer them to just work out of the box.. so I can fiddle with the things I wanted to, and not end up down a (personally) unenjoyable rabbit hole.
No judgment on people who do enjoy it, though! I used to, and maybe I will again at some point.
Having all your mobile traffic routed through AdGuard Home (or PiHole) is a game changer. It's also nice using an exit node through my home network whenever I am on public wifi.
Sure, it's pretty simple. I had WG provided by an Deciso OPNsense router with an automatic VPN profile on most user devices. All of my infrastructure also had PKI. (I moved recently and have yet to set it up again.)
For me, the benefit was simple: I blocked off ssh and Postgres on my production server from the public internet, making it such that these are only accessible from my development machine and other servers. Yet from inside the tailnet I can still talk to Postgres as if it was on LAN. I’m not sure of any easy way to do this without Tailscale.
Any VPN (including WireGuard) would allow you to do the same, and this in indeed how the systems are usually set up. Tailscale just makes the setup a bit easier, though for this use-case not by much, imho.
My primary use of tailscale is as a VPN. I have exit VPSes in 4 different countries, so usually at least one of them is not blocked from or to where I am trying to connect.
> For a long time I didn’t bother with any kind of ACLs within my Tailscale network. […] Then one day a thought hit me. […] That means anyone with access to any of my machines (or who managed to get Tailscale credentials out of one of my apps) would be able to SSH into anywhere else on my network.
I'm a happy Tailscale user but I'll keep saying this whenever Tailscale comes up: We need a way to `tailnet lock` (sign) not just the tailnet nodes but also the tailnet config (ACLs). Otherwise the above scenario of an attacker taking over the entire network is still possible even if you set all ACLs correctly. All it takes is for an attacker to take over the coordination server (to manipulate ACLs) and a single tailnet node. (Which, if you run Headscale, might even be the same machine.)
Until this is fixed I am not going to trust Tailscale with authenticating connections too much and will trade in convenience for defense in depth.
I think I "get" what tailscale is about, what I don't get is how much of it is re-implemented and available out of the box in headscale. I already do most of the things mentioned in the article (from hand-rolled WG, Apache and firewall configurations), so this level of centralised automation and orchestration has some appeal, but I'm not willing to hand over the keys to my entire network to them and would rather keep things in-house.
And on the topic of headscale, some people bring up netbird as an alternative. Netbird gets some immediate sympathy from me as they put lots of emphasis opensource and self-hosted, but I'd be curious to see how they compare for the use-cases described in the article.
I tried using `tailscale funnel` against a dummy server `python -m http.server`, and within 10 seconds the bots started to check for vulnerabilities.
Tailscale warns you about how enabling it will issue an HTTPS certificate which will be in a public ledger. But I wasn't expecting it to be this quick.
I use Headscale, an open source implementation of Tailscale control server. And it doesn't have funnel functionality implemented out of the box, but I use a custom Traefik proxy manager Web UI in which I can expose ports on different Tailnet nodes.
In order to avoid exposing something unnecessarily in the certificate transparency logs, I use a single wildcard certificate, so all the subdomains are not listed anywhere automatically.
I use the same approach for services hosted in the internal subdomain, because I don't want everyone to know what exactly I'm running in my homelab.
Another approach I’ve seen is to route public access from Traefik/nginx through a single Cloudflare tunnel instead, and Tailscale/Headscale can be left for private network and server access.
The traefik box can have the single Cloudflare tunnel , and tailscsle can hang out behind the scenes.
This way tailscale funnel doesn’t need to be public.
There is the self hosted Cloudflare alternative that’s escaping my mind right now too.
Cloudflare also issues certs and logs them in transparency logs. If you do not create a wildcard cert in Cloudflare, your subdomains will leak. And Cloudflare offers free wildcard certs only on the domain root.
Appreciate this super valid consideration.
If services are being exposed for friends and family, using cloudflare tunnels might be a trade off between security or convenience.
If the goal is to ensure security of a home lab online, the less of it that’s discoverable by automated bots, etc, the better.
Yeah, I have mixed feelings about CT (certificate transparency) for this reason. Folks are just consuming the firehose and scanning.
And in this case, if the thing you're funnel'ing is on your residential connection, it basically amounts to you summoning a DDoS.
One (obvious?) tip I'd offer is to put your stuff on high non-standard ports if you can. It'll reduce the amount of connections you get dramatically.
When you care about this, if you're managing your own certificates, you can issue wildcard certificates.
Hmm, yeah, that's a great suggestion, thanks!
Also serve the default website (via IP) from a basically empty self-signed certificate that doesn't give away any domain names or owner details.
You don't have to serve any certificates on the default website. Web server would just fail TLS connection, since it doesn't have a certificate for it.
Not sure if this applies to all web servers, but at least Caddy and a few others support this.
A DoS that will disappear once you close the funnel. Tailscale are proxying the traffic so your public IP isn’t exposed. Your choice of port makes no difference.
Even without CT, services on standard ports will quickly be discovered on IPv4.
> On a computer with a gigabit connection, ZMap can scan the entire public IPv4 address space on a single port in under 45 minutes.
This may discover services, but not hostnames. If the server does not disclose them (e.g. in the certificate used on the IP host), an attacker doesn't have much further to go on.
All the dev servers I've used over the past 10 years come with warnings that they're not security hardened, so I'd be wary of using `tailscale funnel` even though it is awesome to share like that so easily.
Wait, so bots watch for new records added to this HTTPS cert public ledger, then immediately start attacking?
To me that sounds like enabling HTTPS is actually a risk here…
The server was already exposed. All this does is remove obscurity
I wish this trend of “security through obscurity” should mean that all info should just be exposed would die, its silly and lacks basis in reality.
Even within infosec, certain types of information disclosure are considered security problems. Leaking signed up user information or even inodes on the drives can lead to PCI-DSS failures.
Why is broadcasting your records treated differently? Because people would find the information eventually if they scanned the whole internet? Even then they might not due to SNI; so this is actually giving critical information necessary for an attack to attackers.
The issue is not that obscurity per se is bad, but relying _only_ on obscurity is absolute the same as not having any security measures at all.
With the public ledger or not, you will still need to implement proper security measures. So it shouldn't matter if your address is public or not, in fact making it public raises the awareness for the problem. That's the argument.
> relying _only_ on obscurity
Until it gets obscure enough that we start calling it “public-key cryptography”. Guess the prime number I'm thinking of between 0 and 2↑4096 and win a fabulous prize!
If you replace "security by obscurity" with "Kerckhoffs's principle", yes, absolutely!
The problem with using regular everyday obscurity is that it usually has a small state space and makes for terrible security, but people will treat it like it is cleverly hidden and safe from attackers
If I guess the IPv4 you're thinking of between 0 and 2↑32, ready or not, you win a free port scan
As per another comment, we can scan a single port on every public IPv4 address in less than an hour.
Trying every 256bit number gets into a "slightly" larger problem.
> So it shouldn't matter if your address is public or not, in fact making it public raises the awareness for the problem. That's the argument.
Forget about the internet, we've had almost 100 years to prove we can secure identity theft. And the best thing we can do is to keep our SSN's secret -- security through obscurity. Keeping your SSN private reduces your personal attack surface.
We've had 50 years to secure the internet, and yet, we still have zero day attacks. Nuclear submarines try their best to keep their locations a secret? Why? You cannot attack something you cannot see or hear.
Well, this is a bad example, considering public/private key pairs exist,
and work for identity validation,
as long as you don't farm it out to a cheap, know-nothing vendor.
Except we are more on a chess table where we can just trivially probe each cell, unlike the vast volume of the ocean.
A game of battleship is indeed a good analogy!
Just because its a finite space that may eventually be discovered is a poor reason to announce where things are!
Battleship sounds like a good analogy, but is very different because you don't have other options to "secure your ship" besides obscurity. If you had other options, let's say a sonar or moving your ship, they would definitely be used along with obscurity.
Besides, the time to scan the whole board is too time consuming in a battleship game, but scanning the whole internet on the other hand only take a few minutes[1]
[1]: https://github.com/robertdavidgraham/masscan
You're talking IPv4 here, not IPv6. A 24 bit network has 254 addresses in IPv4. A 64bit subnet in IPv6 has 2^64.
If you can scan 1M ipv6's in a second, you can maybe scan 1 subnet in 584,942 years.
So if you're a firewall, and you notice scanning from a particular ip or network, it's easy enough to block them.
Also if you are scanning IPv4, you're not scanning addresses behind the NAT'd routers -- which is also effectively a form of obfuscation. So I would argue it's not the entire internet.
Okay, but we're not talking about that here. This is very much the case of a service being exposed that shouldn't be and relying on obscurity to try and avoid actually getting compromised
ironically I would double down even harder then;
If something was temporary then it’s likely that it wouldn’t have been found in a meaningful amount of time to be exploited.
As an only line of defence it’s not good, but its also not good to hand-deliver your entire personal information to fraudsters and then claim that the systems should be more robust.
If you have a target on your own back thanks to cert transparency logs, it's a bit like closing the barn door late for you to find fault in your own being in Texas when sharpshooters are about. If your only defense was obscurity, your ass is hanging out, and it's no one's fault but your own when you find fault with others for simply saying so.
https://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy
In my original comment I said (I thought) quite clearly that obscurity as your only defence is a terrible idea.
But painting a target on your back is not exactly justified just because hiding yourself isn’t a good defence in of itself.
Obscurity couldn't be anyone's last/best defense, unless it was their only defense, was my point.
In any case, I think we agree.
IME, moving ssh off the standard port reduces bot scanning traffic by >99%. Not only it means less noise in the logs (and thus higher SNR), but also lowers the chance you're hit by spray-and-pray in case there's a zero day in sshd (or any other daemon really).
True, but I hardly open any ssh to the wide world. I would only allow it inside a closed network anyways. HTTP on the other hand _needs_ to be exposed on 80 or 443 (not technically, but in practice)
> IME, moving ssh off the standard port reduces bot scanning traffic by >99%.
Depends on the site I expect. My low value domains get NO ssh attempts on my random ports. The high value ones get a few each week.
You could also always add port knocking or something like that.
If you're going to that level, just put it behind a VPN.
Tailscale is a VPN...
The context of the conversation is that the address becomes publicly visible so you get hit with port scanners and script kiddies looking for vulns. Moving off standard ports does help but many of those are also going to look at ports like 2222 or 8022 and treat them as ssh.
It's not hard to just send something like `nmap -sV -p- <ADDRESS>` (or better, use like rustscan.) and you'll discover those ports and the services.
On the other hand, just install something like knocked and you don't have to do much. Knocking is not a difficult thing to set up.
> Tailscale is a VPN...
And if you use it as a VPN and don't turn on the funnel feature, your service won't be exposed.
> On the other hand, just install something like knocked and you don't have to do much. Knocking is not a difficult thing to set up.
Neither is wireguard.
lol your solution to a problem caused by a feature is "don't use that feature?" LGTM
Presumably wireguard was already being used?
Which is something that makes a notable difference. It’s telling the bots the OP listed are trying Vite endpoints, they’re targeting folks doing short term local web development. Removing obscurity and indicating relative likelihood of still being online is a big shift.
Yes. Yes, of course they do. Check for example https://crt.sh with your domain name to see the glorious public history of everything the certificates tell about your domain.
I don't see why people don't just run their own CAs more for private stuff.
If exposed for others I think the wildcard cert is also what I did, but most tutorials have you issuing certs via ACME for internal or local-only things which doesn't even need to happen.
I personally run my own CA and even setup an ACME server and internal DNS. Nobody knows what I am doing there.
It was common to set up your own CA at one point, especially when DNS management was more manual, However it presented a huge attack surface and was challenging to manage.
A compromised private CA can lead to widespread breaches, affecting various systems and applications that rely on its certificates.
The CAB forum working groups being explicitly prohibited from working on private networks (at least historically) and market incentives also produced a situation where you can't really reduce the blast radius.
ECS1 attacks on AD CS is probably the best publicly documented case for further research.
The happy path is often manageable, but still complex, bland any mistake will result in huge risks.
For me, the value proposition isn’t there. I can get a wildcard domain signed from let’s encrypt and it works out of the box on every device, and you don’t have to deal with the fact that some/many appps will ignore your OS certificate rules.
I use a similar setup, but for anyone following this guide i would not recommend hosting your custom oidc server behind the same tailnet it authorizes.
Any configuration issues will lock you out entirely and you will need to have tailscale support re-enable an oauth provider and its not reversible.
I use an oauth provider to log in to tailscale and keycloak internally as an oidc provider for service to service auth.
I don't know enough about networking as I should, so to plug for my gap in knowledge, I generally prefer to use more comprehensible (to me) forms of security. And a feature like this:
> Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine.
kinda worries me (given also IP spoofing is possible?), compared to SSH keys whose mechanism is more obvious and thus easier to trust.
I definitely like the idea of Tailscale as an extra layer of protection, but I'm not sure I'd loosen existing protections while using it, whereas many Tailscale articles often present it as a panacea for internal-network-over-the-internet security. Are my concerns misplaced?
> kinda worries me (given also IP spoofing is possible?),
It's not, Tailscale authenticates incoming connections. (Note that we're not talking a regular SSH connection to the server's public IP here. You'd connect to the server's SSH daemon through Tailscale.)
Connections are also encrypted with the wire guard protocol using a per-device private key.
I love me some tailscale. But it kills the battery on my phone and it kills resolve.conf every time I boot wsl. I wish I had better luck.
i use zerotier without problems on the phone. yes, they are no longer open source, but source is accessible and it's not worth the effort to switch.
Thanks!
Straight WireGuard to a single point is completely not noticeable.
Neat way to use Tailscale !
I have a similar set-up, without authentication however, relying on Nebula! https://github.com/slackhq/nebula
> It’s a subscription product, but it has an insanely generous free tier that covers basically anything you’d ever want to do as an individual.
Tailscale do have a very nice product, but privacy-conscious users should be aware that you must disable Tailscale's real-time remote collection of your behavior on your “private” network. See KB1011: https://tailscale.com/kb/1011/log-mesh-traffic
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.io). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
It's possible to opt out of this spying on Unix/Windows/Mac clients by starting Tailscale with `--no-logs-no-support` or `TS_NO_LOGS_NO_SUPPORT=true` environment variable (see https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of...), but it is not currently possible to opt out in the Android/iOS clients: https://github.com/tailscale/tailscale/issues/13174
For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326
Also see their privacy policy: https://tailscale.com/privacy-policy#information-we-collect-...
“When you use the Tailscale Solution, we collect limited metadata regarding your device used to access the Tailscale Solution, such as: the device name; relevant operating system type; host name; IP address; cryptographic public key; user agent (where applicable); language settings; date and time of access to the Tailscale Solution; logs describing connections and containing statistics about data sent to and from other devices (“Inter-Node Traffic Logs”); and version of the Tailscale Solution installed.” (emphasis mine)
Anyway, the reason I quoted that part of your post is because Tailscale are using some Fear, Uncertainty, and Doubt tactics here by naming the privacy-preserving option “no-support”, and if you are a free user then you aren't getting support from them anyway, so there should be no downside to keeping your private network private :)
See their blog post about this from last week.
https://tailscale.com/blog/tailscale-privacy-anonymity
# What Tailscale isn't: an anonymity service
Tailscale is a secure connectivity tool that puts the highest value on the privacy of your packets. But we made an intentional choice from day one that we weren't going to try to be an anonymity tool. Quite the opposite in fact! We're an identity-centric network.
Anonymity tools, like Tor, need to be architected very differently. They trade away speed to reduce traceability. They are hard to inspect and diagnose and debug, as a feature. They make enemies, both political and corporate. They are inherently hard to audit and control, by design. In short, they are the exact opposite of what you want your corporate (or even homelab) network to be.
We believe anonymity tools are essential to safe network infrastructure and a free society. But, those tools are made by other people.
…
But if you’re looking for complete anonymity online, Tailscale is not the tool for you. Y'all, we're an identity-centric network with a centralized control plane. You should assume law enforcement can easily find out that you use Tailscale. Tailscale packets are pretty easy to detect, so you can assume they could know, through ISP logs, the shape and size of data you send between different nodes in different places (albeit without knowing the decrypted packet contents). You should assume they can correlate that flow metadata with your login identity.
Open and Close events are not related to identity or anonymity, so that post isn't in itself relevant. It does show that the team is very pragmatic, though.
I get why they capture this data, and by doing so they managed to build an exceptionally great service. But I also understand why one would be uncomfortable with exposing this data.
This isn't relevant to what you were replying to. Parent comment is complaining that there are logs being sent out about what is happening on his private network, he's not expecting anonymity on the internet like Tor (which is what your link describes).
The parent was talking about metadata, which is that same type of data mentioned.
The similarities stop there, both are "metadata".
The issue parent is complaining about is that tailscale sends data about communication data in your _private_ network to their servers. The reply to that is a blog post that clarifies that tailscale is not Tor. Irrelevant, nobody is comparing it to Tor.
That section of the policy simply describes how the system works. It's very valuable information for enterprise customers who are effectively their entire market revenue-wise. Think access logs, intrusion detection, and so on. I do not interpret their policies such that they are processing the information you added emphasis to beyond what is necessary to serve the customer. What evidence do you have to the contrary?
The irony of your post, which brings up Fear Uncertainty and Doubt, is certainly not lost on me. I'm also sure you could just ask apenwarr directly for clarification.
> I do not interpret their policies such that they are processing the information you added emphasis to beyond what is necessary to serve the customer. What evidence do you have to the contrary?
Respectfully, you are failing to appreciate the full scope of the problem. It doesn't matter what Tailscale do with the data. The log contents don't matter at all, only the fact that a network connection was made. Every network connection you make creates metadata about you, and the Internet itself — the path between me and Tailscale's logging endpoint — is always listening.
Think what conclusions can be drawn about a person's behavior from a log of their network connections. Encryption doesn't matter, because we're just talking about metadata; each connection's timestamp, source, destination, and port. Think about the way each additional thing-which-makes-network-requests increases the surveillance value of all the others.
Straight away, many people's NTP client tells the network what OS they use: `time.windows.com`? Probably a Windows user. `time.apple.com`? Probably Mac or iOS. `time.google.com`? You get the idea. Yeah, anyone can configure an NTP client to use any of those hosts, but the vast vast majority of people are taking the default and probably don't even know what NTP is.
Add a metadata point: somebody makes a connection to one of the well-known Wi-Fi captive portal detection hosts around 4PM on a weekday? Maybe somebody just got home from school. Captive portal detection at 6PM on a weekday? Maybe somebody just got home from work. Your machines are all doing this any time they reconnect to a saved Wi-Fi network: https://en.wikipedia.org/wiki/Captive_portal#Detection
Add a metadata point: somebody makes a network connection to their OS's default weather-widget API right after the captive-portal test, and then another weather-API connection exactly $(DEFAULT_INTERVAL} minutes later? That person who got home is probably still home.
Anyway, you get the point that this stuff adds up! The problem with Tailscale is that its default behavior exposes metadata about entire additional classes of traffic in addition to all the examples I just mentioned that my devices were already leaking. Tailscale would have me start telling the Internet “hey I'm here and doin' stuff!” every time I read or write any file on my NAS, every time I use Steam Link remote play over LAN, every time I SSH or RDP into any of my other machines.
The free “Personal” tier is limited to only 3 users but 100 devices, so it's normal and expected to set it up the client on any and every computer you own: https://tailscale.com/kb/1154/free-plans-discounts#personal-...
My behavior would be exposed to every layer of service provider along the way: my ISP, my ISP's ISPs, the cloud provider Tailscale use to host their surveillance endpoint, Tailscale themselves if they so choose, whatever creepy secret spy implants we're not allowed to know about. No thanks! If you want to be private, you must be silent.
> My behavior would be exposed to every layer of service provider along the way: my ISP, my ISP's ISPs, the cloud provider Tailscale use to host their surveillance endpoint,
Maybe I'm missing something here but I'd guess that data is encrypted and not a free for all of open data that any old ISP could snoop on. If not that'd be a serious issue.
Not to say that you don't have some good points. Even just the pattern and timings of that data being sent could be exploited. Also TS would still have that full data.
Though I'd have to study the details. Do they aggregate and then send it at regular intervals, etc? In the end would it be that much worse than what Apple, Google, Microsoft collect?
> Maybe I'm missing something here but I'd guess that data is encrypted and not a free for all of open data that any old ISP could snoop on.
Yes, you are missing the entire point. You are talking about data. I am talking about metadata — data about data. The contents of each log request are a total red herring. Just pretend that the encrypted log messages are a single bit, just a way to increase a counter that “something has happened” on a person's Tailnet.
The encrypted log message structure does tell Tailscale “this particular machine on the Tailnet talked to this other particular machine on the Tailnet at this time”, and one should assume Tailscale decrypt and interpret those details, but what I'm talking about is the ability for any part of the network path to interpret those log connections without decrypting them as “somebody is using their Tailnet right now in any capacity”, and when, and from where, and the ability to combine that new class of metadata with all the other metadata our modern OSes are constantly generating.
> Do they aggregate and then send it at regular intervals, etc?
This is already addressed in my original comment. Again, see KB1011: https://tailscale.com/kb/1011/log-mesh-traffic
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.com). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
“This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
“real-time events”!!
e: Recommended reading:
- https://kieranhealy.org/blog/archives/2013/06/09/using-metad...
- https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-...
Eh, as a network administrator you want the netlogs on by default and you very clearly onboard everyone to the network with a memorable warning to do their personal browsing over some other interface. You've usually got at least some minimal audit requirement on any network with high value stuff on it.
It's probably not great that someone trying to use the free sample product lands in the same netlogging regime as the work network default, but I suspect thats more about allocation of attention and priority which understandably goes to the companies that make up approximately all of their business. Keeping the free sample product around after its long bern clear "this is for work computers" is just one of those things. The "no support" suffix on a setting is not to me the smoking gun you make it out to be, and I'm pretty hardcore in my attitudes about surveilance.
I agree it's the wrong default for a purely personal user, but TailScale has enough "good faith actor" points with me that I'll give them the benefit of the doubt on malicious/evil dragnet surveilance ambitions. What could they possibly want with the data of a group of people who are by construction not spending money on a VPN? They'd be storing it at a loss.
> What could they possibly want with the data of a group of people who are by construction not spending money on a VPN? They'd be storing it at a loss.
This is the exact point where our conclusions diverge.
Why are they sending themselves so much "useless" data-intensive logs by default, from their non-paying clients that accounts for roughly ~95% of the userbase and from a profitable business perspective, largely ineligible for troubleshooting support? For me, the only logical conclusion is that the data is valuable to them.
As someone who also cares about privacy, hear are a few things that IMO suggest that free customers' logs are a part of their business model:
* Their documentation has plenty of references to security, but no references to privacy outside of the privacy policy.
* They have all but eliminated any revenue stream from average user when they made an unsolicted announcement that they upgraded their free plan to allow 100 devices and 5 users.
* The content they sponsor for marketing/advertising seems targeted for consumers instead of networking professionals. I don't see Cisco and Palo Alto Networks sponsoring every Linux/self-hosting podcast or YouTube channels for example.
* Even the flag-name for turning off logging is mild deterrent based on what you will lose (`--no-support`) as opposed to being neutral '--no-logging' or truly descriptive like most FOSS companies that are not pushing an ulterior motive such as '--no-analytics'.
* logs cannot be disabled for phones
* In my experience, disabling logs was perhaps the only thing that was not configurable through the GUI
I'm into privacy and still relatively new on the networking scene thanks to setting up OpenWrt on my router. Am I correct that when tailscale updates/hijacked resolv.conf, subsequent DNS resolution is passed onto them on visited websites even when tailscale is not being used? No, they can't "read" your traffic, but if I understand things right, they know every website you visited and for how long, which is more than enough data for a rich marketing profile. That was my takeaway before I jumped ship for a self-hosted solution.
My understanding is that they have the holy grail of data because they are getting all of your LAN, WAN and mobile network traffic. I'm not aware of (m)any companies whose business model allows access to all three? It's like if your ISP and your Mobile Network had a baby on your local server, and that baby reports every website you visit.
> Am I correct that when tailscale updates/hijacked resolv.conf, subsequent DNS resolution is passed onto them on visited websites even when tailscale is not being used?
I think you're incorrect in the default settings, even when tailscale is active.
From the docs, last paragraph under Global nameservers https://tailscale.com/kb/1054/dns#global-nameservers
By default, your tailnet's devices use their local DNS settings for all queries. To force clients to always use the nameservers you define, you can enable the Override DNS servers toggle.
> I think you're incorrect in the default settings
What mac-attack is correct about is that by default, `tailscaled` sets itself as the only DNS resolver and proxies all DNS requests to your non-Tailscale nameservers. Citations:
- https://tailscale.com/kb/1381/what-is-quad100#dns-resolver
“`100.100.100.100` or Quad100 is a special Tailscale IP address […] that provides essential local services. It operates similarly to the localhost address (`127.0.0.1`) but serves only Tailscale-specific services. These services include a DNS resolver.”
“One of the services provided by Quad100 is a DNS resolver running on port 53 (1100.100.100.100:531). A DNS resolver is a service that translates IP addresses to hostnames like `google.com` or `macbook.tailnetname.ts.net`. Quad100 is a stub resolver, similar to systemd-resolved, except with extra features.”
- https://tailscale.com/blog/sisyphean-dns-client-linux
“The upcoming Tailscale 1.8 release implements all of the above [other DNS managers], which should hopefully make DNS on Linux just work, no matter how your machine is choosing to do it.”
- https://tailscale.com/kb/1235/resolv-conf
“Tailscale overwrites `/etc/resolv.conf` when MagicDNS is enabled in the tailnet”
“Tailscale tries to interoperate with a number of other DNS managers before resorting to overwriting `/etc/resolv.conf`.”
- https://tailscale.com/kb/1081/magicdns
“Tailnets created on or after October 20, 2022 have MagicDNS enabled by default.”
It does say “While Quad100's DNS resolver operates locally without logging, forwarded requests might be logged by configured nameservers.”, but the fact remains that the Tailscale software is very aggressive about taking over all DNS resolution on your system. Once that is done, the option of whether or not `tailscaled` overrides your default nameservers can be configured remotely without you knowing it's happening!
https://tailscale.com/kb/1054/dns#tailscale-dns-settings
I'm split on this. According to your links, it tries to cooperate with the system resolver. If it can't find a way to do it, then yeah, it kinda has to replace it.
Of course, they could put this much more front and center in the docs, so that if you're running some funky setup and know what you're doing, you should be able to easily do it - which you probably can with the `--disable-dns thing`. But putting it in a prominent spot in the docs could help to not overlook this.
I've just checked the setup on a machine running systemd-networkd and resolved, and resolv.conf wasn't touched. It only added a specific dns setup for the tailscale0 interface, which only covers my tailnet name and ips. It doens't even show as a fallback or whatever in the global section.
> the option of whether or not `tailscaled` overrides your default nameservers can be configured remotely without you knowing it's happening!
I mean, there's two situations. Either we're talking about a "pro" environment, where corp vpns taking over your local network config, as much as I hate it, isn't exactly news. Then there's the personal plans users, in which case, if the DNS changes without you knowing, you probably have way bigger problems.
> In my experience, disabling logs was perhaps the only thing that was not configurable through the GUI
A lot is not configurable from the GUI. Announcing routes for instance (out of my head, there are other switches I set on tailscale) Or exit nodes
Logging everyones network data/metadata would likely be illegal under employment law in Norways. Other European countries may have same/similar rules. GDPR may also apply. So be careful with how broadly you apply that default.
This comment should really be much higher.
You do get support if you're a free user, it's just best effort and via e-mail only.
One useful additional aspect to Tailscale that I've not seen mentioned so far, is the integration with Mullvad.
Using that you can get the benefit of their network servers, appearing just as standard Tailscale exit nodes, which is handy if you need to geo-shift traffic at all.
> One useful additional aspect to Tailscale that I've not seen mentioned so far, is the integration with Mullvad.
Indeed, this saved my butt earlier this year when I was at Qatar airport and they tried to MITM most of my connections (including to mullvad.net). Luckily, they didn't MITM tailscale.net, so I could log in, enable Mullvad, and thereby secure my entire traffic.
Are you sure they weren't just trying to redirect you too a capture portal?
Can you give more details for the un-initiated? What is Mullvad? How and why would you use this feature?
Is it to use it as a kind of VPN to make traffic "appear" from any country, and, eg, watch Netflix?
Indeed, Mullvad is a VPN provider (https://mullvad.net/en) they provide several privacy focused services including a VPN and a browser.
Using Mullvad (or other VPN products) allows someone to make their traffic originate from specific countries, which can allow for testing to see what it'll look like from different countries, or to access content which is restricted to people in a specific country. It can also allow for bypassing age check restrictions that apply to specific geographies (e.g. the UK)
Integrating it with Tailscale makes it easier to use (if you're an existing tailscale user) as instead of having to install and manage a separate product, it integrates with your existing tailscale setup, allowing you to dynamically choose a mullvad exit node in a give country, and then your Internet traffic will appear from that country.
(that all sounds somewhat ad-like but I'm not in any way affiliated with Tailscale or Mullvad apart from being a user (Tailscale did give me a nice hoodie one time for writing a blog though ))
Sounds a bit like a fancier ngrok.
Accidentally wiring everything to everything else sounds kind of scary.
There's 1 or 2 things I wouldn't mind securely exposing to the internet (like Plex) but nothing I need so desperately while I'm out and about that I'd even want to take that risk.
Sounds like this is just for self-hosting?
Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine. This is particularly handy when you SSH from a phone, as proper credential management is a bit of a nightmare there.
this has me worried. i would not want that. i use zerotier, not tailscale, but the principle is the same. i have my laptops and my phone connected to my servers. given that all of those machines are already on the internet, connecting them into a virtual network does not add any risk in my opinion. (at least as long as you don't use features like the above). all i get is a known ip address for all my devices, with the ability to connect to them if they have an ssh server running. when i am outside the primary benefit is that i can tell which devices are online.
This feature isn't enabled by default.
this is for teams where you don't want to create passwords or keep track of ssh keys for everyone by hand. it greatly simplified our server usage as we can simply ssh user@machine and it just works. you can create access controls for it as well.
> Sounds a bit like a fancier ngrok.
Well, yes and no.
You can use it like ngrok, and I'm sure you could configure wireguard and ngrok to give you something similar to what Tailscale does, but Tailscale does it out of the box, with polished and well built client and server apps.
I'm no infra guy, I'm just a former front-end eng, but it gives me the confidence to expose media centres and file servers etc to "the wild" without it being public.
Using Jellyfin to watch content from my home server on my iPad while I'm away from home is as "easy" as Disney or Netflix with Tailscale, just installed the clients and servers and .. voila?
I was an infra guy early in my career, and I'm still savvy, and I still prefer using Tailscale. It's very polished and reliable.
But personally, I'm past the point of wanting to fiddle with things like this and would much prefer them to just work out of the box.. so I can fiddle with the things I wanted to, and not end up down a (personally) unenjoyable rabbit hole.
No judgment on people who do enjoy it, though! I used to, and maybe I will again at some point.
It's pretty handy to setup an exit node on your home network so that when you go abroad you can still watch your streaming services too!
More importantly you can use it to access your bank or other services which often block non-US IPs. It's saved me a few times in the last year or two.
Having all your mobile traffic routed through AdGuard Home (or PiHole) is a game changer. It's also nice using an exit node through my home network whenever I am on public wifi.
Plex already supports remote access via UPnP. https://support.plex.tv/articles/200289506-remote-access/
To me WireGuard is safer than exposing services directly to the internet.
Sure, it's pretty simple. I had WG provided by an Deciso OPNsense router with an automatic VPN profile on most user devices. All of my infrastructure also had PKI. (I moved recently and have yet to set it up again.)
Tailscale is able to hole punch in scenarios where UPnP is disabled (just good practice) as well as many NAT environments.
I wrote a POC for using Tailscale serve and funnel similarly to ngrok here:
https://github.com/jaxxstorm/tgate
if you guys love Tailscale, perhaps take a look at NetBird, an open source solution, which also has a commercial offer. Really recommend this one
For me, the benefit was simple: I blocked off ssh and Postgres on my production server from the public internet, making it such that these are only accessible from my development machine and other servers. Yet from inside the tailnet I can still talk to Postgres as if it was on LAN. I’m not sure of any easy way to do this without Tailscale.
Any VPN (including WireGuard) would allow you to do the same, and this in indeed how the systems are usually set up. Tailscale just makes the setup a bit easier, though for this use-case not by much, imho.
I just wish Tailscale let you use taildrop to directly send files to other people
My primary use of tailscale is as a VPN. I have exit VPSes in 4 different countries, so usually at least one of them is not blocked from or to where I am trying to connect.
I’ve been experimenting with different ways of using WireGuard but hadn’t heard of the header based authentication Tailscale does. Interesting stuff.
Are people using this because they don’t want to configure dynamic DNS and NAT?
Yes, but more so as a VPN. It's much easier than any traditional VPN to setup.
> For a long time I didn’t bother with any kind of ACLs within my Tailscale network. […] Then one day a thought hit me. […] That means anyone with access to any of my machines (or who managed to get Tailscale credentials out of one of my apps) would be able to SSH into anywhere else on my network.
I'm a happy Tailscale user but I'll keep saying this whenever Tailscale comes up: We need a way to `tailnet lock` (sign) not just the tailnet nodes but also the tailnet config (ACLs). Otherwise the above scenario of an attacker taking over the entire network is still possible even if you set all ACLs correctly. All it takes is for an attacker to take over the coordination server (to manipulate ACLs) and a single tailnet node. (Which, if you run Headscale, might even be the same machine.)
Until this is fixed I am not going to trust Tailscale with authenticating connections too much and will trade in convenience for defense in depth.
I think I "get" what tailscale is about, what I don't get is how much of it is re-implemented and available out of the box in headscale. I already do most of the things mentioned in the article (from hand-rolled WG, Apache and firewall configurations), so this level of centralised automation and orchestration has some appeal, but I'm not willing to hand over the keys to my entire network to them and would rather keep things in-house.
And on the topic of headscale, some people bring up netbird as an alternative. Netbird gets some immediate sympathy from me as they put lots of emphasis opensource and self-hosted, but I'd be curious to see how they compare for the use-cases described in the article.