xyzio

Archive for the ‘Programming’ Category

Authenticate with BackBlaze B2 and list buckets with Python

with one comment

Authenticate with BackBlaze B2 and list buckets with Python.

First to authenticate with BackBlaze B2:


import requests
from requests.auth import HTTPBasicAuth

#Credentials from BackBlaze B2 manager
key_id          = 'b2_key_id'
application_key = 'b2_application_key'

#Contact authorization server and check response
path   = 'https://api.backblazeb2.com/b2api/v1/b2_authorize_account'
result = requests.get(path, auth=HTTPBasicAuth(key_id, application_key))
if result.status_code != 200:
    print 'Error - Could not connect to BackBlaze B2'
    exit()

#Get results and add authorization token to headers
result_json = result.json()
account_id  = result_json['accountId']
auth_token  = result_json['authorizationToken']
api_url     = result_json['apiUrl'] + '/b2api/v1'
api_session = requests.Session()
api_session.headers.update({ 'Authorization': auth_token })

Now use api_session to interact with B2. For example, download the list of buckets like this:


#Construct the API url
url = api_url + '/b2_list_buckets'

#Use the api_session parameter to get a list of buckets
bucketInfo = api_session.post(url, json={'accountId': account_id})

#Convert datastream to json
jsonBucketInfo = bucketInfo.json()

#List id and name
for bucket in jsonBucketInfo['buckets']:
    bucketId = bucket['bucketId']
    bucketName = bucket['bucketName']

    print bucketId, bucketName

More details about tokens in the json are on BackBlaze’s list_buckets API page.

Written by M Kapoor

December 23, 2019 at 8:05 pm

Posted in Programming

Tagged with ,

Authenticate with BackBlaze B2 and get file URLs using Python

leave a comment »

Authenticate with BackBlaze B2 and get the URLs of files in a bucket.

First to authenticate with BackBlaze B2:


import requests
from requests.auth import HTTPBasicAuth

#Auth information from Backblaze
key_id = 'key_id'
application_key = 'application_key'

#Authenticate
path = 'https://api.backblazeb2.com/b2api/v1/b2_authorize_account'
result = requests.get(path, auth=HTTPBasicAuth(key_id, application_key))
if result.status_code != 200:
    print 'Error - Could not connect to BackBlaze B2'
    exit()

#Read response
result_json = result.json()
account_id = result_json['accountId']
auth_token = result_json['authorizationToken']
api_url    = result_json['apiUrl'] + '/b2api/v1'
download_url = result_json['downloadUrl'] + '/file/'
api_session = requests.Session()
api_session.headers.update({ 'Authorization': auth_token })

Now get bucket contents and assemble URL. Sample code for bucket ID and bucket name is here.


#Initialize
bucketId = 'bucket_id_from_b2'
bucketName = 'name_of_bucket'

params = {'bucketId': bucketId}
urls = set()

#Loop for as long as a nextFile exists
while True:
    #Construct api call, execute, and read back information
    url = api_url + '/' + 'b2_list_file_names'
    fileList =  api_session.post(url, json=params)
    jFileList = fileList.json()

    #Loop through files and construct url
    for file in jFileList['files']:
        urls.add(download_url + bucketName + '/' + file['filename'])

    #Check for next file and break if it doesn't exist
    startFileName = jFileList['nextFileName']
    if startFileName == None:
        break
    else:
        #continue If next file exists
        params['startFileName'] = startFileName
        

Written by M Kapoor

December 23, 2019 at 7:17 pm

Posted in Programming

Tagged with ,

Setting up a golang website to autorun on Ubuntu using systemd

leave a comment »

Here is how to set up a website to auto-launch and restart on failure on Ubuntu using systemd. I’m using a simple site written in Go and compiled into an executable called gosite – I got sample Go code from Jeffrey Bolle’s page.

Simple golang website code

package main

import(
    "net/http"
    "log"
    "os"
)

const resp = `Simple Web App
Hello World!`

func handler(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte(resp))
}

func main() {
    http.HandleFunc("/", handler)
    err := http.ListenAndServe(":8080", nil)

    if err != nil {
        log.Println(err)
        os.Exit(1)
    }
}

Setting up systemd

Once the code is ready and tested, lets move on to setting up systemd.

Create and open a file in the /lib/systemd/system folder:

 vim /lib/systemd/system/gosite.service

Edit the file with your parameters to match the contents below:

[Unit]
Description=A simple go website
ConditionPathExists=/home/user/bin/gosite

[Service]
Restart=always
RestartSec=3
ExecStart=/home/user/bin/gosite

[Install]
WantedBy=multi-user.target

Enable the service using systemctl:

root@hostname:/home/user# systemctl enable gosite.service
Created symlink from /etc/systemd/system/multi-user.target.wants/gosite.service to /lib/systemd/system/gosite.service.

Start the service:

service gosite start

Observe the service is running:

root@hostname:/home/user# ps aux | grep gosite
root 27413 0.0 0.2 827776 5012 ? Ssl 15:50 0:00 /home/user/bin/gosite

Written by M Kapoor

June 14, 2016 at 4:49 pm

Posted in Programming

Tagged with , , ,

Hacking Planet Atwood with Python and AWS

leave a comment »

Peter Atwood is a hobbyist who creates limited edition pocket tools and puts them up for sale on his site at Planet Pocket Tool.  His tools are fairly popular and being limited are hard to acquire.

Peter posts the sales randomly and the tools generally sell out within a few minutes of being listed.  The best way to capture a sale is to periodically check his site and get alerted when a sale is in progress.  This write-up is about automating the check and sending an alert via SMS and email using Python.

Blogspot publishes a RSS feed for their blogs.  I wrote a simple function to use feedparser to grab the datetime of the first item in the feed and compare it against the previous version.  I send out an alert if the datetime of the first item is different from the one I have saved.

#Get the blog entry
feed = feedparser.parse('http://atwoodknives.blogspot.com/feeds/posts/default')

#Figure out the publish time of the first entry
firstEntryPubTime = time.strftime('%Y%m%d%H%M%S', feed.entries[0].published_parsed)

#The newest post time did not match the previously saved post time
#We have a new post
if firstEntryPubTime != currentUpdateTime:

    #Save the new first blog entry time
    setCurrentUpdateTime(firstEntryPubTime)
    url = feed.entries[0].link
    subject = 'Atwood: ' + feed.entries[0].title
    body = url + feed.entries[0].summary

    #Send an alert
    send_alert(subject, body, url)

else:
    print "No Update"

Now, to send the alert we leverage Amazon Web Services and BOTO the AWS Python interface. Amazon has a service called Simple Notification Service or SNS. SNS is a push service that lets users push messages in various formats like SMS and email.

Getting started is simple. First create a topic to which people can subscribe using create_topic. Then subscribe your phone number, email address, and any other form of communication using subscribe. Now you are all set.

def send_alert(message_title, message_body, message_url):
    #Connect with boto using the AWS token and secret key
    c = boto.connect_sns('token','secret_key')
    topicarn = "arn:aws:sns:us-east-1:TopicName"
    #Publish or send out the URL of the blog post for quick clicking
    publication = c.publish(topicarn, url, subject=url[:110])
    #Close connection
    c.close()

I set up this script on a server at DigitalOcean and ran it periodically using cron. I was able to get to the buy link for most of Atwood’s sales with this methodology and eventually bought a Fancy Ti Atwrench. While nice and well made, it is definitely not worth what Peter Atwood charges for it.

Written by M Kapoor

June 10, 2015 at 11:58 pm

Driving Adafruit’s RGB Matrix with Python on the BeagleBone Black

leave a comment »

I wrote this code to drive the Adafruit 32×64 RGB Matrix using a Beaglebone Black and Adafruit’s IO Python library.  The code is super simple – it writes a static color background to the top and bottom halves of the display through the shift registers and triggers the latch.  It then iterates through the addresses so the LEDs for that row turn on and then goes to the next address.  This should create a background color through persistence of vision.

Along the way I discovered:

The Python Library isn’t fast enough to drive the RGB matrix to provide a persistent display.  There is flicker and when updating colors, you see each row turn on and off.  You literally see the delay.  There are ways to speed up the writes by using the PRU or direct IO writes.  But I should not have to do that on a 1GHz processor.

The RGB matrix doesn’t light up unless you continually change the address.  This was really annoying when I was figuring things out.  I did not find this documented anywhere on the web except on rayslogic’s RGB writeup.

It appears the BeagleBone Black’s IO pins cannot push enough current to turn on the input schmitt triggers.  I spent a lot of time being frustrated because I couldn’t get the green and blue LEDs to light up on my board.
Finally I noticed the green LEDs were barely on and had a brightness gradient from low to high address.  My guess is the address pin was barely turning on which meant 0 was the most common address – hence the higher brightness on the lower rows.  I got around this by plugging the G(reen) input directly into the 5V pin on the BeagleBone Black.  Of course this now meant the green LEDs are always on!

Adafruit charges too much for the RGB panels.  For example, the seller kbellenterprises on ebay has a 16×32 RGB matrix for $16 including shipping.  Adafruit has the same panel for $25 and you pay for shipping.

Adafruit charges too much for male-to-male jumper wires.  The seller funny-diy on ebay sells 40 for $1.80 with shipping while Adafruit charges $4+shipping for their so-called “premium” wires.  Adafruit needs to rename them to “premium-priced“.

My Python Code:
https://bitbucket.org/xyzio/rgbmatrix/src/master/bb_python/logic.py

Sources:
http://www.rayslogic.com/propeller/programming/AdafruitRGB/AdafruitRGB.htm
https://learn.adafruit.com/setting-up-io-python-library-on-beaglebone-black/overview

 

Written by M Kapoor

April 3, 2015 at 4:12 pm

Non-Arduino Adafruit OLED Display Library

leave a comment »

Adafruit sells a really nice 16×2 OLED display and they even have an Arduino library for it.  However I only have a Atmega32 and Arduino does not run on the Atmega32.  Rather than spending $20 or so for an Arduino, I decided to port their library to AVR.  It wasn’t too difficult since Arduino is just AVR C++ behind the scenes.

Re-writing the Adafruit library was straightforward.  I hardcoded PORTA as the I/O port but some additional code could make that generic.  I had to re-write the digitalRead and digitalWrite functions to read and write out of PORTA instead of the Arduino pins.  The delayMicroseconds command is replaced by the AVR _delay_ms() macro.

The trickiest part was figuring out how to implement the print function.  After some digging around I found it in the Arduino code at \hardware\arduino\cores\arduino\Print.cpp.  The print function calls a write function that iterates through the character string which then calls our native OLED write on each character.  This seems seems a little roundabout but I guess that is how they chose to implement it.

Adafruit 16x2 OLDED display

Links:

My No-Arduino code:
https://bitbucket.org/xyzio/avr-code/src/master/characterOLED/

Adafruit Arduino library:
https://github.com/ladyada/Adafruit_CharacterOLED

Youtube demo:

Written by M Kapoor

December 26, 2014 at 3:02 pm

Browsing through Amazon’s Top 10 Best Sellers

leave a comment »

This project lets you browse through Amazon’s various product categories and view the top 10 sellers for that category.  You can browse down into sub-categories as well as up into parent categories.

I wrote this to learn about Amazon’s Product Advertising API.  Using the API is simple, especially with the sample code provided by Amazon.  Amazon returns a lot of information, so the slightly tricky part was figuring out how to parse the XML to get only the information I need.

 

Amazon Top Ten Browser

 

Written by M Kapoor

September 10, 2014 at 4:37 pm

RSS Feed Creator with hosting

leave a comment »

Need a way to get your audiobooks, open courseware, videos, or other content onto your podcast player? The RSS Feed Maker will take your URLs and convert them into a RSS document and give you a URL to give to your podcast player.

The RSS feed is created with Argotic and the files are hosted on Amazon’s S3 file hosting service.

 

 

 

 

 

 

 

 

 

 

 

Written by M Kapoor

September 8, 2014 at 11:59 pm

AnalogDesert – A Free Android DigitalOcean App

leave a comment »

AnalogDesert is a simple open-source ad-free Android app to check the status of your Digital Ocean Droplets.

Download it here:
https://play.google.com/store/apps/details?id=com.xyzio.analogdesert

Initial Setup:
Go to the settings menu and enter your Digital Ocean Client ID and API Key.
To get your Digital Ocean Client ID and API Key, log into your account and click on API.

Do you see bugs or want more features? Contact Me!

Analog Desert supports the following:
Droplets – Create, Destroy, Details, Disable/Enable backups, Password reset, Power On/Off, Reboot, Rebuild, Rename, Resize, Restore, Shutdown, Snapshot and Visit.
Images – View all global & personal images, destroy, transfer
Sizes – View sizes
Domains – View domains, domain information, create domains

 

Source:
https://bitbucket.org/xyzio/analogdesert

 

 Screenshots:

Analog Desert New Droplet Menu for Digital Ocean

New Droplet

Droplets Menu

Destroy Droplet Digital Ocean

Destroy Droplet

Written by M Kapoor

August 11, 2013 at 3:06 pm

Posted in DigitalOcean, Programming

Tagged with , , ,

Free Online Wordsearch Puzzle Generator

leave a comment »

A Wordsearch puzzle is a puzzle that contains words hidden in a grid of text. It is a fun activity used to teach children spelling and to help memorize information like country capitals.

Interested in creating your own Wordsearch puzzle? Then try out the Free Online Wordsearch Puzzle Generator at http://xyzio.com/Projects/WordSearch/Default.aspx.

This Word Search generator was written in C#/ASP.NET and generates PDFs using the iTextSharp library.

Sample Page:

Free Online Wordsearch Example

Wordsearch Example

Written by M Kapoor

June 29, 2013 at 12:23 am