ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Find MangoLassi Post Views Per Thread

    IT Discussion
    script screen scraping
    2
    4
    304
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scottalanmillerS
      scottalanmiller
      last edited by

      Following @dbeato's script that helps you to download an entire website, I'll follow up with a simple one that helps you to grab every thread on ML and tells you how many views it has. This thread takes a long time to run, as it reads a lot of data. You can just output to a text file and then manipulate as you like. Or you can easily make a list of threads that you care about and track things in that way. It is easy to change it just a little to make it useful for you in different scenarios.

      #!/bin/bash
      for i in {1..20000}; do echo $i " " $(curl --location --referer ";auto" --netrc -s -D - http://mangolassi.it/topic/"$i" | grep human-readable-number | grep -v topic | cut -d'>' -f2 | cut -d'<' -f1); done
      
      1 Reply Last reply Reply Quote 3
      • scottalanmillerS
        scottalanmiller
        last edited by

        Here is some sample output...

        1   20790
        2   2518
        3   1928
        4   9380
        
        1 Reply Last reply Reply Quote 1
        • dbeatoD
          dbeato
          last edited by

          @scottalanmiller said in Find MangoLassi Post Views Per Thread:

          and tells you how many views it has. This thread takes a long time to run, as it reads a lot of data. You can just output to a text file and then manipulate as you like. Or you can easily make a list of threads that you care about and track things in that way. It is easy to change it just a little to make it useful for you in different scenarios.

          I need to see how to use Curl to download the site, as it might be better than wget.

          scottalanmillerS 1 Reply Last reply Reply Quote 0
          • scottalanmillerS
            scottalanmiller @dbeato
            last edited by

            @dbeato cURL is really good for on the fly processing. WGET is more focused on getting the whole site down to disk.

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post