Bitcoin-data manipulaton and plotting in python

 

plot.png - https://i0.wp.com/raw.githubusercontent.com/Altoidnerd/bitcoin-price/master/plot.png?resize=840%2C474&ssl=1
I wrote a python mini-api to manipulate and visualize the price of bitcoin historically using the coinbase version 1 api.

bitcoin-price (python module)

Get bitcoin price data from the coinbase API and parse it into an easily manipulated form.

>>> from fast_dump_v22 import *
>>> get_page(1)

Will return all the data on page 1 of coinbase’s bitcoin price API (this is the latest data).

You can almost always turn on optional debug statements

>>> get_first_N(3, show=True)
... fetching first 3

You can get all the price data that coinbase has

>>> get_all(show=True)
... getting page 1
... getting page 2
... getting page 3
... getting page 4
... getting page 5
... getting page 6
... getting page 7
... getting page 8
... getting page 9
... etc ...

All the ‘get_*’ functions return a price_data string, which is interlaced timestamps and prices littered with newlines and commas. You can print them to see what is going on more clearly:

>>> print(get_page(1))
2015-09-11T06:44:04-05:00,241.14
2015-09-11T06:34:04-05:00,240.8
2015-09-11T06:24:04-05:00,240.75
2015-09-11T06:14:05-05:00,240.68
2015-09-11T06:04:04-05:00,240.83
2015-09-11T05:54:05-05:00,240.92
2015-09-11T05:44:04-05:00,240.64
2015-09-11T04:34:04-05:00,241.27
2015-09-11T04:24:04-05:00,240.73
...

Turn on the optional show switch for printing large vectors

>>> prices(get_page(11), show=True)
... returning 11000 prices in specified range ...
['239.9',
'239.9',
'239.4',
'239.77',
'239.33',
'239.99',
'239.81',
'240.28',
'240.4',

You can use prices(data)[k] and timestamps()[j] to return the kth price in data, or the jth timestamp in data.

>>> data = get_page(1)
>>> prices(data)[4]
'241.2'
>>> prices(data, index=4)
'241.2'

are two equivalent ways of returning only the 4th price in the requested range (in this case, page 1). This also works for timestamps.

>>> timestamps(get_page(1)+get_page(2))[1166] == timestamps(get_first_N(2), index=1166)
True

This shows the expressiveness of this module. In general:

>>> prices(get_page(2)) == parse(get_page(2))[0]
True

prices() and timestamps() are just functions that return a parsed() object having a specific index, or indices.

>>> parse(get_page(1)+get_page(2)+get_page(3))[0] == prices(get_first_N(3))
True
>>> parse(get_page(2)+get_page(3))[0] == prices(get_range(2,3))
True

The parse() function is there to manually control the outputs instead of just getting prices, or timestamps

>>> x = parse(get_page(1))
>>> x[0][0]
'241.2'
>>> x[0][1]
'241.14'
>>> x[1][1]
'2015-09-11T04:34:04-07:00'

As you can see, parse(price_data)[0][k] returns the kth price in the list. Indices [1][k] return the kth timestamp.

The parse() function takes care of some weird edge cases:

>>> get_first_N(3) == get_page(1)+get_page(2)+get_page(3)
False
>>> parse(get_first_N(3)) == parse(get_page(1)+get_page(2)+get_page(3))
True
>>> x = get_page(1)
>>> y = get_range(2,7)
>>> prices(get_first_N(7)) == prices(x+y)
True

In general,

OPERATOR( get_page(1) + get_page(2) + ... + get_page(k) ) == OPERATOR(get_first_N(k))

where OPERATOR is parsed(), prices(), or timestamps(). We also know prices() can obviously display and return ranges of values. When returning large vectors, you can verify their length by setting show=True. The “show” parameter is optional for all get_* functions and provides some information about the operation being performed.

>>> print( prices(get_first_N(11), show=True) )
... returning 11000 prices in specified range...

since each page is a thousand pairs of values (timestamp, price).

>>> len(prices(get_page(2))
1000
>>> prices(get_page(2)) # returns a long list
['230.11',
'229.64',
'230.04',
'229.71',
'229.69',
'229.74',
'229.92',
'229.43',
'229.43',
'229.41',
...

fast_dump_v1* are older versions that are somewhat different. They are designed to store the fetched data in the .data directory. This in v2*, this was abandoned in favor of stdout redirection.
// 2015 altodinerd.com

Getting started with Bitcoin the “right way” – encrypted wallets, bitcoin core, and linux

A brief introduction to getting started with bitcoin on a linux distribution

For the impatient: Skip to step 1

Bitcoin is one of the most interesting technologies we have seen emerge in the past ten years. And it confuses everyone who sets out to use it.  This post is my definitive guide to beginning your journey with bitcoin the right way.

tuxbig

To me, the right way includes, but is not limited to:

  • Running bitcoin on a linux machine.  My reasons for this are too many to really list here, but suffice it to say that since linux is widely regarded as the most secure operating system available, and the usage of bitcoin involves being your own bank, using bitcoin on linux is the best way to ensure your coins stay safe.
  • Using the bitcoin-core wallet software.  There are other wallets available.  You actually don’t even need a wallet software to own bitcoins or even spend bitcoins.  However, if you want to understand how bitcoin works under the hood – which you should, because you are going to be your own bank, you should use this software.
  • Using the command line to do bitcoin things. Actually, using both the GUI and the command line is the way to go.  This tutorial (this is part 1 of 2) will be a command-line-centric description because afterall, I’m trying to show you the right way to do things.

But that’s just, like, your opinion man.

It is. This is my blog.

You shouldn’t necessarily trust me or anyone – bitcoin isn’t about trust. But if you google “altoidnerd”, you’ll find that I have been around the bitcoin eco-system for quite some time. That’s all I can tell you.

Screen Shot 2015-09-10 at 8.31.28 PM

There are just so many wallets available – why bitcoin-core?  Because I have been using it for years, I have a very good method of keeping my coins safe, and that is what I am sharing with you today.  Use it- it is THE wallet.

But altoidnerd, I don’t want to run linux. I like windows / OSX

I am a linux and an OSX user.  So please understand the scope of my knowledge includes these operating systems. If you want to use OSX, most of this tutorial will make sense with minor changes perhaps.

Screen Shot 2015-09-10 at 8.32.59 PM

If you want to use windows, that’s ok, I’m not going to judge you. Bitcoin-core is available for windows as well. Please understand however, I am not a windows user, so I will not be able to give you “the definitive bitcoin windows guide”. You can still read on, however, since you can apply much of this tutorial to usage of bitcoin-qt’s debug window instead of the command line.

Step 1: Download and install bitcoin-core

Navigate to bitcoin.org’s official download page and select the bitcoin-core distribution for your operating system. And do choose linux (tgz).  Do that because you should be running linux.  But if you aren’t, even though you should be, choose the distribution for your operating system. Extract and install the client.






The “right way”

To really take control of things, and understand what is happening, you’re going to want to run bitcoin software from the command line. Though it isn’t really necessary to download the thing from the command line, I’m going to describe that here because this tutorial is the first installment of a start to finish command line approach. For a sneak peak of what I mean, see an earlier post where I described how to use bitcoin-core to generate QR codes without having to trust shady ass websites.

Go to your home directory and create a new directory just for bitcoin.  This is not a necessary step, but once again, this is my “right way” tutorial and I will explain reasons for this later on.  Enter the directory, and download the tarball with the “wget” command.  The extract the tarball like so:

    cd ~

    mkdir bitcoin

    cd bitcoin

    wget 'https://bitcoin.org/bin/bitcoin-core-0.11.0/bitcoin-0.11.0-linux64.tar.gz'

    tar -xzvf bitcoin-0.11.0-linux64.tar.gz

bitcointerm1

 

Sweet.  Now change directory to where the binaries are,

    cd bitcoin-0.11.0/bin/

and launch “bitcoin-qt” (qt means its the GUI version).

./bitcoin-qt

It’s going to show you the dialogue box below and ask you if you’d like to use the default directory. Do it. Click “OK”.

 

default

 

You’ll see a friendly startup screen if you’ve done things right!

bitcointerm2

 




 

Step 2: Encrypt your wallet, and wait a few days

Bitcoin-core is called a “full-node” implementation, which means the first time you start it on your machine, it’s going to download every single bitcoin transaction that has ever taken place.  Wait until the blockchain is synced to start making transactions.

In the mean time, you need to set a passphrase for your wallet.  This is super important.  In the upper left corner of bitcoin-qt, go to settings -> Encrypt Wallet, and set a very strong passphrase.

encrypt

This will make it impossible for anyone to send bitcoins from your wallet without entering your password:

PASSWORD

Just make sure you:

    • Make your password extremely long. I don’t know if there is a length limit in the bitcoin-core code or not. Regardless, seriously push your personal boundaries on what you think a safe password is. Make it super duper long, and impossible to guess. Wise men have discussed password strength to a great extent. Pick a strong password. 20+ characters. Go for 30. No 40. Just make it long, and keep it an absolute secret.
    • Never, ever lose or forget this password. If you lose or forgot your password, your coins will be unspendable forever. Neither you, nor anybody on earth, nor God himself will be able to spend your coins. Your coins will be effectively lost.





Picking a strong passphrase makes security so, so easy

Once you encrypt your wallet, you can copy and store wallet.dat everywhere (I explain wallet.dat in great detail in “Step 3”). I literally have copies of my main wallet.dat on 6 or 7 computers, because come hell or highwater, I will always be able to find a copy of my wallet.dat file. I suggest you do the same. Here is why strong encryption and insane redundancy works:

  • If your password is ridiculously long, you can be very lazy and downright cavelier about where you keep your wallet.dat, because even if someone finds the wallet file, they wont be able to do a damn thing with it. They won’t be able to spend your coins without your passphrase.
  • This means that you can make copies of wallet.dat and store them like literally everywhere, on all your machines, three times over. In the cloud. On your mom’s computer. Hell, theoretically speaking, you can post your encrypted wallet.dat on the internet and just about everyone will have a copy, and be unable to spend your coins, because they don’t know your password. Someone remind me to put some coins into an address, and post the encrypted wallet online to prove my point.

    Bottom Line

    If you encrypt your wallet like a boss, you never forget your password, you never tell anyone your password, and you copy the file to everywhere you possibly can imagine, you will never lose your coins. And nobody will be able to steal them from you. Seriously, nobody. It is computationally impossible.

    Step 3: Understand what you just did on your computer

    Let’s break it down this way. Here are some things that just happened when you launched bitcoin-qt for the first time:

    • A hidden directory ~/.bitcoin was created. It contains among other things the single most important thing ever: a file called wallet.dat. This file is critical. ~/.bitcoin/wallet.dat contains all of your private keys. wallet.dat is life. When you see in headlines in the news like “Frustrated gentleman quite upset having lost a usb jump drive containing 100,000 BTC“, it means he doesn’t have a copy of wallet.dat, so neither he, nor anyone on earth, nor God himself can spend his coins anymore.
    • Your computer started downloading the blockchain. The blockchain is a record of every bitcoin transaction ever to have taken place. This will take a few days. As of today, the blockchain is about 42 gigabytes. Make sure you have space for it.

    Do not delete your wallet.dat file. Just don’t. If you want to start a new wallet, instead of deleting wallet.dat, just rename it to something else, like wallet.dat.old, restart bitcoin-qt, and it will create a new wallet for you. Keep your wallets people. Keep them good.

    wallet

    Notice how I have a file called wallet.dat.default? That’s because I never delete a wallet. When I installed bitcoin on this computer, I moved the wallet.dat that stores my coins into ~/.bitcoin/, but first I renamed the existing file so I can keep it. Because, why not? I’m telling you – don’t delete a file called wallet.dat, because mistakes happen, and its better to have hundreds of files named wallet.dat.* than it is to lose your coins.

In my next post, I will describe some of the features of the bitcoin-qt debug window, bitcoind and its helper bitcoin-cli.  You can do interesting things, like import new keys, or convert keys to qr-codes.




If you have ay questions, please feel free to comment or contact me.
If you liked this post, you can donate bitcoins to me here: 12gKRdrz7yy7erg5apUvSRGemypTUvBRuJ
Learn about the author here:
http://www.phys.ufl.edu/~majewski

How import a bitcoin address private key into breadwallet (iOS) with a QR code using bitcoin-core for Ubuntu / Linux

On encoding private keys from Bitcoin-core as QR codes for use in paper wallets, other wallet software, etc …

Note: Breadwallet will not import the private key upon scanning the qr code. It will offer to transfer the funds in that address to your breadwallet keyring.

Breadwallet, an awesome open source iOS bitcoin SPV client, allows you to import a private key for use within breadwallet.  So, if you’re a user of bitcoin-core, and want to be able to spend some of your funds you have there with breadwallet, you’ll need to create a QR code of your private key to scan it with breadwallet.

Install qrencode

This tutorial uses qrencode, an open source string to QR converter. The source can be found here: https://github.com/fukuchi/libqrencode/

Alternatively, you can install qrencode on ubuntu/debian through repositories

sudo apt-get install qrencode

Then, as of bitcoin-core version 0.11.0 you’ll need to run bitcoind rather than the GUI bitcoin-qt. If you’re using an old version of bitcoin-core that does not have the bitcoin-cli tool, you can get the latest version of bitcoin-core and extract it without having to verify the blockchain again or mess with wallet.dat – your ~/.bitcoin directory will remain unchanged.

Get the newest version of Bitcoin-core


wget 'https://bitcoin.org/bin/bitcoin-core-0.11.0/bitcoin-0.11.0-linux64.tar.gz'
tar -xzvf bitcoin-0.11.0-linux64.tar.gz
cd bitcoin-0.11.0/bin

You should see the executables for the bitcoin-core suite here.
image1



Run bitcoind and create your QR

Be sure you quit any versions of bitoin-qt or bitcoind that are already running, then start the bitcoin daemon

./bitcoind &

The & operator makes sure bitcoind detaches from your terminal so you can keep going.

Now you are ready to get a QR code for your desired address. Is your address is “addr”, you can generate a QR code of your private key without ever showing your private key on screen in plain text. Just do:

./bitcoin-cli dumpprivkey "addr" | qrencode -o ~/Desktop/Key.png

Or better yet, if you want to create a new address and QR-encode its private key,

addr=$(./bitcoin-cli getnewaddress)
./bitcoin-cli dumpprivkey $addr| qrencode -o
~/Desktop/$addr.privkey.png
echo "private key dumped for: "$addr

The QR code should appear on your desktop. Below is a screen shot of me doing an example where I create a new address, and get a QR for its private key.  Enjoy!

image0



Note that if your wallet is password protected (which it really should be), you’ll need to first do

./bitcoin-cli walletpassphrase "your_password" 100

where the argument 100 means the wallet will allow all actions (like spending, or dumping keys) for 100 seconds. That should be plenty of time.




When should I sell my Bitcoin Mining hardware? Bitcoin Mining Hardware Resale Value vs Projected Return

It’s interesting to look back at these “old” numbers from November 2013!

20140605-062924.jpg

When should I sell my Bitcoin Mining hardware? Bitcoin Mining Hardware Resale Value vs Projected Return

When should I sell my Bitcoin Mining hardware? Bitcoin Mining Hardware Resale Value vs Projected Return.

 
Donate Bitcoins