Google is encrypting its search result URLs. I'm still recovering all of them in one pass. — Suganthan
suganthan.com
Google now hides the destination URL on every search result behind an encrypted blob. I took the encryption apart, and I can still read every ranked URL off the page in one pass.

About a week or two ago, I started noticing that I could no longer copy the clean links for search result hits on Google. I thought it was maybe some quirk of a browser I was using, but today when trying to copy a link to a Youtube video, I realized it was actually rolled out globally (it seems).

Basically instead of getting a
https://youtube/. com/watch?v=-ed6UeDp1ek link now through google, (spaced out link to avoid AutoMod) you will get
https://www.google.com/goto?url=CAES<rest-of-encrypted-blob-containing--versionByte-keyID-cipherText>

The article describes the change in technical detail, and mentions a fix, but the author has kept it private for now so Google doesn’t patch it up (like they probably will or have for this workaround relying on the “Translate Page” feature with a different Browser Language selection [Twitter Link, Aug 27]).

Why it will probably be patched:

Surprised there’s no post about it here already.
Only Hackernews article I could find (4 days ago):
https://news.ycombinator.com/item?id=49506338

Edit: It seems to matter a lot whether your network is deemed “spammy” or problematic by Google for the encrypted URLs to kick in, even if you’re logged out. Funnily enough my google searches went back to normal a few hours after I posted this. So a logged out session might get the regular, unencrypted links if your IP network has a good rating on Google’s end. Mine often runs into captchas and the like, so it makes sense why I’d be targeted by this new change.
Then again, I just tested on my mobile cellular on a logged out search, and I still have the goto’s. Can’t really control that.

  • chunes@lemmy.worldEnglish
    37·
    16 days ago

    So google has no problems unleashing their bots on others but they don’t want to get scraped themselves?

    That’s the sort of shit that should get your company disbanded in any functioning civilization.

    • deleted@lemmy.worldEnglish
      1·
      16 days ago

      I like it so much, and it’s my go-to search tool. Sometimes it struggles to identify official websites, but that only happens in about 1 out of every 100 searches.

      I wish it had image search though.

    • onionsinmypores@sh.itjust.worksOPEnglish
      41·
      16 days ago

      I wasn’t aware of Qwant. It seems to build its own index instead of relying on other providers (great), and has proper backing to be a viable competitor in something so big (the French government seems to endorse it; along with institutional use in EU parliament?).
      I just tested it and unfortunately got a 403 however (FORBIDDEN). Though I was behind an endpoint of a very popular VPN for some testing, that sucks regardless.

      Actually, before I could post this, I tried the search refresh and it worked somehow.
      To test again I switched to another server of the same VPN provider (same selected country) and tried searching. Same error, which didn’t go away after doing the refresh. So they might have some sort of spam prevention measure which eases itself automatically, but I feel like it could definitely be implemented better if that’s the case.

      Edit: Yep, if you sit on that 403 page for 5-6 minutes without doing anything and then hit the relaunch your search button, things work as intended afterwards.

      • kuiskaaja@lemmy.mlEnglish
        28·
        16 days ago

        turn off your adblocker

        edit: lmao why so many downvotes? this is what qwant does when you use an adblocker

        • onionsinmypores@sh.itjust.worksOPEnglish
          2·
          16 days ago

          I tried without uBlock Origin on my end and it did indeed work. I incorrectly assumed it was my vpn without properly verifying. Though CactusEcho said there were problems without the adblocker so it may be inconclusive.

  • _haha_oh_wow_@sh.itjust.worksEnglish
    2·
    16 days ago

    Glad we have stuff like Startpage and Kagi, which are both better than Google anyways. There’s also Qwant, but it’s not as good as Kagi or Startpage.

    • punkfungus@sh.itjust.worksEnglish
      1·
      16 days ago

      I just tried a test search with Qwant and Startpage. On mobile they returned the exact same results (which were good), which suggests that Startpage is using Qwant there. On desktop though, Startpage was hot garbage. Not a single relevant result on page one, with nearly half being links to Facebook and Instagram. Hard pass on that.

      I tested this on mobile too. Opened a Firefox private browsing window, switched to desktop mode and tried the search term on Startpage again. Same trash results as desktop. It’s like they’re actively telling desktop users to piss off.