Originally posted on: http://geekswithblogs.net/michelotti/archive/2013/07/24/fun-with-the-chrome-javascript-console-and-the-pluralsight-website.aspxI’m currently working on my
third course for Pluralsight. Everyone already knows that Scott Allen is a “dominating force” for Pluralsight but I was curious how many courses other authors have published as well.
The Pluralsight Authors page - http://pluralsight.com/training/Authors – shows all 146 authors and you can click on any author’s page to see how many (and which) courses they have authored.
The problem is: I don’t want to have to click into 146 pages to get a count for each author. With this in mind, I figured I could write a little JavaScript using
the Chrome JavaScript console to do some “detective work.” My first step was to figure out how
the HTML was structured on this page so I could do some screen-scraping. Right-click
the first author - “Inspect Element”. I can see there is a primary <div> with a class of “main” which contains all
the authors. Each author is in an <h3> with an <a> tag containing their name and link to their page: This web page already has jQuery loaded so I can use $ directly from
the console. This allows me to just use jQuery to inspect items on
the current page. Notice this is a multi-line command. In order to use multiple lines in
the console you have to press SHIFT-ENTER to go to
the next line: Now I can see I’m extracting data just fine. At this point I want to follow each URL. Then I want to screen-scrape this next page to see how many courses each author has done. Let’s take a look at
the author detail page: I can see we have a table (with a css class of “course”) that contains rows for each course authored. This means I can get
the number of courses pretty easily like this: Now I can put this all together. Back on
the authors page, I want to follow each URL, extract
the returned HTML, and grab
the count. In
the code below, I simply use
the jQuery $.get() method to get
the author detail page and
the “data” variable that is in
the callback contains
the HTML. A nice feature of jQuery is that I can simply put this HTML string inside of $() and I can use jQuery selectors directly on it in conjunction with
the find() method: Now I’m getting somewhere. I have every Pluralsight author and how many courses each one has authored. But that’s not quite what I’m after – what I want to see are
the authors that have
the MOST courses in
the library. What I’d like to do is to put all of
the data in an array and then sort that array descending by number of courses. I can add an item to
the array after each author detail page is returned but
the catch here is that I can’t perform
the sort operation until ALL of
the author detail pages have executed.
The jQuery $.get() method is naturally an async method so I essentially have 146 async calls and I don’t want to perform my sort action until ALL have completed (side note: don’t run this script too many times or
the Pluralsight servers might think your an evil hacker attempting a DoS attack and deny you). My C# brain wants to use a WaitHandle WaitAll() method here but this is JavaScript. I was able to do this by using
the jQuery Deferred() object. I create a new deferred object for each request and push it onto a deferred array. After each request is complete, I signal completion by calling
the resolve() method. Finally, I use a $.when.apply() method to execute my descending sort operation once all requests are complete. Here is my complete console command: 1: var authorList = [],
2: defList = [];
3: $(".main h3 a").each(function() {
4: var def = $.Deferred();
5: defList.push(def);
6: var authorName = $(this).text();
7: var authorUrl = $(this).attr('href');
8: $.get(authorUrl, function(data) {
9: var courseCount = $(data).find("table.course tbody tr").length;
10: authorList.push({ name: authorName, numberOfCourses: courseCount });
11: def.resolve();
12: });
13: });
14: $.when.apply($, defList).then(function() {
15: console.log("*Everything* is complete");
16: var sortedList = authorList.sort(function(obj1, obj2) {
17: return obj2.numberOfCourses - obj1.numberOfCourses;
18: });
19: for (var i = 0; i < sortedList.length; i++) {
20: console.log(authorList[i]);
21: }
22: });
And here are
the results:
WOW! John Sonmez has 44 courses!! And Matt Milner has 29! I guess Scott Allen isn’t
the only “dominating force”. I would have assumed Scott Allen was #1 but he comes in as #3 in total course count (of course Scott has 11 courses in
the Top 50, and 14 in
the Top 100 which is incredible!). Given that I’m in
the middle of producing only my
third course, I better get to work!
