rsync a file using a ssh .pem public key
Using a public SSH key with rsync to transfer a single file. The key is the -e option which modifies the SSH command used by rsync.
rsync -a ./localfile.jpg user@hostname.com:/home/user/localfile.jpg -e 'ssh -i keypath.pem'
Return pdf to browser in a C# razor page using itext7
Sample code to return a PDF file to the browser in a Razor c# page using itext7.
using System;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.RazorPages;
using Microsoft.Extensions.Logging;
using iText.Kernel.Pdf;
using iText.Layout;
using iText.Layout.Element;
using System.IO;
public IActionResult OnPostManipulatePdf(String dest)
{
MemoryStream ms = new MemoryStream();
PdfWriter writer = new PdfWriter(ms);
PdfDocument pdfDoc = new PdfDocument(writer);
Document doc = new Document(pdfDoc);
writer.SetCloseStream(false);
Paragraph header = new Paragraph("header");
doc.Add(header);
doc.Close();
byte[] byteInfo = ms.ToArray();
ms.Write(byteInfo, 0, byteInfo.Length);
ms.Position = 0;
FileStreamResult fileStreamResult = new FileStreamResult(ms, "application/pdf");
//Uncomment this to return the file as a download
//fileStreamResult.FileDownloadName = "Output.pdf";
return fileStreamResult;
}
Source:
https://stackoverflow.com/questions/1510451/how-to-return-pdf-to-browser-in-mvc
PlugIn Board – An AVRISP breakout board for prototyping with the AVR microcontroller
In the past I would connect AVR microcontrollers to the AVRISP programmer by using individual wires to connect the 6×2 IDC connector to the protoboard. There was also the hassle of wiring up a 5v supply. So I designed the PlugIn Board to quickly connect the AVRISP programmer to AVR microcontrollers on a protoboard.
Layout
Looking at the PCB Board Layout below, the PlugIn board takes the AVRISP 2×3-pin IDC connector (ISP1) and maps to a 0.1-inch in-line header that plugs into a protoboard. In addition it can power the AVR. Unregulated power can be fed through a 2.1mm DC jack or screw-down terminal connector(J1) which is connected to a 7805 regulator(IC2) to provide 5V to the AVR and other circuits.
A few other features in the PlugIn board is it has the recommended 10kOhm pull-up resistor(R1) for reset and the pinout matches the ATMEGA32 programming and power pins.
PCB Board Layout
Future Improvements
If I were to do another iteration of the PlugIn Board I’d add the recommended decoupling circuitry for the ADC with a separate ground plane as outlined in the Atmel AVR design considerations app notes.
While not recommended due to the long signal path, I’d also include a header for an external clock. It would allow the user to swap out crystal oscillators or use an external clock source without fiddling around on the protoboard.
PCB Board Files
The PCB is designed using Eagle. The board and schematics are available on my BitBucket at https://bitbucket.org/xyzio/circuits/src/master/PlugIn%20Board/
Final Product
I had the board fabbed at OSH Park. It works well and greatly simplifies programming AVR microcontrollers.
Generating Charts with Python and matplotlib as Base64 images for embedding in HTML webpages
This is code used to create the charts for my ‘Compare Expense Ratios‘ microsite. It uses matplotlib to generate a chart which is converted to a Base64 to be embedded directly into a webpage. This way there is no need to save an intermediate image.
#Inputs are a dictionary of lists that contains points to plot (points), the title, and the x and y-axis labels.
def mathPlotLib(points, title, xlabel, ylabel):
import matplotlib
matplotlib.use('Agg')
from matplotlib import pyplot
import base64
import cStringIO
import natsort
#The two funds are the keys to the dicitonary
fundOrder = []
for fund in natsort.natsorted(points.keys()):
#Plot the points and keep track of the order they are added for the legend
pyplot.plot(points[fund])
fundOrder.append(fund)
# Add the legend, title, and x,y labels.
pyplot.legend(fundOrder)
pyplot.title(title)
pyplot.xlabel(xlabel)
pyplot.ylabel(ylabel)
# Convert the image as png as a byte-string object
my_stringIObytes = cStringIO.StringIO()
pyplot.savefig(my_stringIObytes, format='png')
# Seek to the beginning of the file and encode it as Base64
my_stringIObytes.seek(0)
b64png = base64.b64encode(my_stringIObytes.read())
# Add the html wrapper to embed it as base64
html = '<img src="image/png;base64,' + b64png + '" />'
# Clean up
my_stringIObytes.close()
pyplot.clf()
pyplot.close()
return html
Automate build and load of Hugo sites to Amazon S3 using Rclone with Python
This was the build and load flow for my Hugo site before I gave up on Hugo and moved back to WordPress. The site was generated using Hugo and pushed to Amazon s3 using rclone.
The code assumes the hugo and rclone executables, and the hugo root directory are in the same folder.
In this example the hugo directory is called codepearls.xyzio.com.
import os, shutil # s3 auth info s3SecretAccessKey = 's3_secret_key' s3AccessKeyId = 's3_access_key' # Hugo folder and bucket name. sitepath is path to the hugo public folder. path = 'codepearls.xyzio.com' sitepath = os.path.join(path, 'public') #Remove the files from the previous build by deleting the public folder if os.path.exists(sitepath): shutil.rmtree(sitepath) # Run hugo from root directory cmd = 'hugo.exe -s ' + path os.system(cmd) # Run rclone from root directory to sync /public to s3 and the appropriate args to encrypt the files and enable public read cmd = 'rclone.exe sync ' + sitepath + ' s3:' + path cmd += ' -v --s3-secret-access-key ' + s3SecretAccessKey cmd += ' --s3-access-key-id ' + s3AccessKeyId cmd += ' --s3-acl public-read' cmd += ' --s3-server-side-encryption AES256' cmd += ' --s3-provider AWS' cmd += ' --s3-region us-west-2' os.system(cmd)
Split an Excel file’s tabs into CSV with C#
Split an Excel file into csv, with each tab going into a separate csv file, in the background using C#.
using System.IO;
using Microsoft.Office.Interop.Excel;
Microsoft.Office.Interop.Excel.Application app = new Microsoft.Office.Interop.Excel.Application();
//Run in background
app.DisplayAlerts = false;
//Open excel file
Workbook wb = app.Workbooks.Open("path_to_excel.xls");
//Iterate through sheets in the excel file
foreach (Worksheet sheet in wb.Worksheets)
{
string sheetName = sheet.Name;
//Output file name
string outputFilepath = sheetName + ".csv";
//Save the sheet as CSV
sheet.SaveAs(outputFilepath, XlFileFormat.xlCSVWindows);
}
//Clean-up
wb.Close();
app.Quit();
Open a zip archive and iterate through its files with Python
Open a file compressed as .zip and iterate through its files line by line with Python.
import zipfile
#Path to the zip file
zipfilepath = 'path_to_zip_file'
#Read in zip file
zip = zipfile.ZipFile(zipfilepath)
#Iterate through files in zip file
for zipfilename in zip.filelist:
#Read contents of the file
filecontents = zip.read(zipfilename)
#Break up contents into list and process
for line in filecontents.replace('\r\n', '\n').split('\n'):
print line
Open and print a .gz.bz2 file with Python
Open and print a file that is compressed with gzip and then with bzip2, i.e a gz.bz2 file, with Python.
import sys import bz2 import gzip from cStringIO import StringIO # .gz.bz2 File is given as commandline argument filename = sys.argv[1] gzbzfilename = filename #Read in the bz2 data o = open(gzbzfilename, 'rb') gzbzdata = o.read() #Decompress the bz2 data gzdata = bz2.decompress(gzbzdata) #Next, gunzip the gzip file and read out the file pointer f = gzip.GzipFile(fileobj=StringIO(gzdata)) file_content = f.read() f.close() #Print out the file contents print file_content
Use ffmpeg To Create a Timelapse From Pictures With Music
Create a timelapse movie from images using ffmpeg with background music.
Images must be sequentialy numbered starting with the value given for the start_number argument.
Command:
ffmpeg -framerate 12 -start_number 15 -f image2 -y -i "G%7d.jpg" -i music.mp3 -c:v libx264 -preset fast -c:a mp3 -shortest -movflags +faststart "output_filename.mp4"
Options Breakdown:
| Option | Description |
|---|---|
| framerate 12 | Number of images to show per second |
| start_number 15 | Start number of the first image |
| -i “G%7d.jpg” | ffmpeg formatted file name |
| -i music.mp3 | Background music filename |
| -shortest | Truncate video to shortest of timelapse or music |
| -movflags +faststart | Put file index at beginning to speed up online streaming |
Convert magnet link to torrent file with Python and add additional trackers
With rtorrent it is easier to have it pick up torrent files from a directory. This code converts a magnet link into a torrent file with Python and then inserts additional trackers to the torrent file.
import urllib
import HTMLParser
#This is the magnet link to modify
magnetLink = 'magnet:?xt=urn:btih:b54a3ba68fd398ed019e21290beecc9dda64a858&dn=wikipedia_en_all_novid_2018-06.zim&tr=udp%3a%2f%2ftracker.mg64.net'
#Here is a list of trackers to insert into the torrent
trackers = {'list','of','trackers'}
#Convert HTML special characters to escaped normal characters
magnetLink = urllib.unquote(magnetLink).decode('utf8')
magnetLink = HTMLParser.HTMLParser().unescape(magnetLink)
#Split out the trackers from the magnet link and save for later
magnetsplit = magnetLink.split('&tr=')
base = magnetsplit[0]
magnetTrackers = magnetsplit[1::]
#Add the trackers from the magnet link to our list of trackers
for magnetTracker in magnetTrackers:
trackers.add(magnetTracker)
#Add the trackers
magnetLink = base
for tracker in trackers:
magnetLink += '&tr=' + tracker
#Create the torrent file name - it is named after the magnet hash
magnetName = magnetLink[magnetLink.find("btih:") + 1:magnetLink.find("&")]
magnetName = magnetName.replace('tih:','')
torrentfilename = 'meta-' + magnetName + '.torrent'
#Write the magnet link to the torrent file
with open(torrentfilename, 'w') as o:
linkstr = u'd10:magnet-uri' + str(len(magnetLink)) + u':' + magnetLink + u'e'
linkstr = linkstr.encode('utf8')
o.write(linkstr)
xyzio 