<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Richard K Miller &#187; BYU</title>
	<atom:link href="http://richardkmiller.com/category/byu/feed" rel="self" type="application/rss+xml" />
	<link>http://richardkmiller.com</link>
	<description></description>
	<lastBuildDate>Wed, 01 Feb 2012 15:26:50 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Reminiscing about Provo411.com and Scraping the Course Catalog</title>
		<link>http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog</link>
		<comments>http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog#comments</comments>
		<pubDate>Thu, 08 Oct 2009 17:58:11 +0000</pubDate>
		<dc:creator>Richard K Miller</dc:creator>
				<category><![CDATA[Business]]></category>
		<category><![CDATA[BYU]]></category>
		<category><![CDATA[Entrepreneurship]]></category>
		<category><![CDATA[Main]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Tech]]></category>

		<guid isPermaLink="false">http://richardkmiller.com/?p=780</guid>
		<description><![CDATA[One of my first web development projects and biz partnerships with Brian Stucki was Provo411.com. We were roommates at BYU and conceived of a website where students could share events &#8212; parties, concerts, football games, etc. We were already in &#8230; <a href="http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<div class='microid-094c99b227bd0f07e51e726ad993de96274495c3'><p>One of my first web development projects and biz partnerships with <a href="http://www.brianstucki.com/blog/">Brian Stucki</a> was <a href="http://www.provo411.com/">Provo411.com</a>. We were roommates at BYU and conceived of a website where students could share events &#8212; parties, concerts, football games, etc. We were already in our beds for the night when the idea came, but we couldn&#8217;t go to sleep before buying the domain. I think it was the first domain I ever bought. It was September 2002.</p>
<p>I developed a calendar in PHP and wrote a few scripts to scrape <a href="http://byucougars.com/">byucougars.com</a> and retrieve the sports schedules. I also developed a <a href="http://en.wikipedia.org/wiki/Wireless_Markup_Language">WML</a> app so Brian and I could add events to the calendar from our pre-iPhone mobile phones. I recall being at a party in south Provo, in a former dental office, and using my Nextel phone to add the party to Provo411. If you go back far enough, you can see <a href="http://www.provo411.com/2003/10">events on the calendar</a>. My brother Alan did the artwork.</p>

<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/byu' title='BYU'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/BYU.jpg" class="attachment-thumbnail" alt="BYU" title="BYU" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/concert' title='Concert'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Concert.jpg" class="attachment-thumbnail" alt="Concert" title="Concert" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/dance' title='Dance'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Dance.jpg" class="attachment-thumbnail" alt="Dance" title="Dance" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/football' title='Football'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Football.jpg" class="attachment-thumbnail" alt="Football" title="Football" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/live_band' title='Live_Band'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Live_Band.jpg" class="attachment-thumbnail" alt="Live_Band" title="Live_Band" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/meal' title='Meal'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Meal.jpg" class="attachment-thumbnail" alt="Meal" title="Meal" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/soccer' title='Soccer'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Soccer.jpg" class="attachment-thumbnail" alt="Soccer" title="Soccer" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/talk' title='Talk'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Talk.jpg" class="attachment-thumbnail" alt="Talk" title="Talk" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/theater' title='Theater'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Theater.jpg" class="attachment-thumbnail" alt="Theater" title="Theater" /></a>
<a href='http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/volleyball' title='Volleyball'><img width="30" height="30" src="http://richardkmiller.com/wp-content/uploads/2009/10/Volleyball.jpg" class="attachment-thumbnail" alt="Volleyball" title="Volleyball" /></a>

<p>I always wanted Provo411.com to have a course schedule alert system. Perhaps students would pay $3 to receive an email or SMS alert when hard-to-get classes had an opening. It shouldn&#8217;t have been hard technically, but the <a href="http://saas.byu.edu/classSchedule/schedule.php">publicly available course catalog</a> isn&#8217;t updated in real-time. I could have scraped the authenticated course catalog on Route Y, but BYU might have objected and it&#8217;d be a fragile business model.</p>
<p>My brother Michael recently came home from <a href="http://www.youtube.com/watch?v=TbwT4j-mLdw">his mission</a> and started school at <a href="http://www.csn.edu/">CSN</a>. The business classes he wanted were full, so I put the old &#8220;course schedule alert&#8221; idea to the test with some new tools &#8212; Ruby and Mac OS X&#8217;s speech. Here&#8217;s what I came up with:</p>
<pre class="brush: ruby; title: ;">
#!/usr/bin/env ruby

# a list of course call numbers to check
call_numbers = %w{ 46405 46407 46409 46411 46415 46413 53252 53254 53256 53258 53260 53262 53268 53270 53272 53274 46423 46435 53276 46443 }

# auth_token obtained via Firefox+TamperData while my brother logged into CSN
auth_token = &quot;123456789012345&quot;

say &quot;Checking&quot;

call_numbers.uniq.sort.each do |call_number|
    c = `curl -si -d CONVTOKEN=#{auth_token} -d AUDITT=N -d CALLT=#{call_number} -d CONTINUE=Continue &quot;https://bighorn.nevada.edu/sis_csn/XSMBWEBM/SIVRE04.STR&quot;`
    print &quot;Call number #{call_number}: &quot;
    if (c =~ /&lt;p class=&quot;p5&quot;&gt;([^&lt; ]+)&lt;br\/&gt;/m)
        if $1.strip.empty?
            puts &quot;May have openings\n&quot;
            3.times {say &quot;Michael, class number #{call_number} may be open!&quot;}
        else
            puts &quot;#{$1.strip}\n&quot;
        end
    else
        puts &quot;could not find message&quot;
        say &quot;Help. I cannot access the C S N website.&quot;
        return
    end
    sleep 5
end

# Ouput an audible message via Mac OS X's speech function
def say(message)
    `say &quot;#{message}&quot;`
end
</pre>
<p>We set this to run every 15 minutes on the living room iMac, and we turned up the volume. Every 15 minutes we could hear &#8220;Checking&#8221; from the computer. A few hours later we heard the script announce that a class had opened up. Michael, I&#8217;m still waiting for my $3.</p>
</div>]]></content:encoded>
			<wfw:commentRss>http://richardkmiller.com/780/reminiscing-about-provo411-com-and-scraping-the-course-catalog/feed</wfw:commentRss>
		<slash:comments>9</slash:comments>
		</item>
	</channel>
</rss>

