Microsoft

Python SQL Server driver on Linux

So you have packaged your SQL monitoring and maintenance routines into a web server and demonstrated it all works from your computer. Impressed they ask for it to be put on a proper server – a Linux box. 5 years ago this would have involved using unsupported 3rd party drivers and who ran internal Linux servers anyway. Now the request seems almost reasonable although you will have to jump through more hoops than you would with Windows.

First off I’ll assume you are using the pyodbc module. On Linux this will require a C compiler. If you have chosen a minimal install then you’ll need to install them. This can be done with the following command (depending upon the flavour)

Redhat (Centos/Fedora)
sudo yum groupinstall 'Development Tools' -y
sudo yum install python-devel unixODBC-devel -y

Debian (Ubuntu)
sudo apt-get install build-essential -y
sudo apt-get install python-dev unixodbc-dev -y

With this done you can now pip install pyodbc. The pyodbc module is a wrapper around the native system drivers so you will need to install a suitable unixodbc driver. Microsoft have produced an official unixODBC driver since 2012 and it has been regularly maintained since. Installation instructions for v13 can be found on this blog post.

With pyodbc and unixodbc set up all you need to change in your actual code is the driver on the ODBC connection string to ‘ODBC Driver 13 for SQL Server’ and away you go. As a quick test, the following example will establish a connection and return the servername through a SQL query.

import pyodbc
cnxnstr = "Driver=ODBC Driver 13 for SQL Server;Server=<yourserver>;Uid=<yourusername>;Pwd=<yourpassword>;database=<yourdatabase>"
cnxn = pyodbc.connect(cnxnstr)
cursor = cnxn.cursor()
cursor.execute("SELECT @@SERVERNAME")
result = cursor.fetchall()
for row in result:
    print(row)
cursor.close()
cnxn.close()

Virtual environments in Visual Studio

A virtual environment in Python is a folder with everything needed to set up local configuration isolated from the rest of the system. This allows you can have modules installed locally which are different or do not exist in the global Python configuration. If you have used Node.js then you can think of virtual environments as npm default way of working – creating a local install of a package rather than a global one (pip’s default).

If you have multiple versions of Python installed on your machine then you can also specify which version of Python the virtual environment should use. This gives you ability to test your code against multiple versions of Python just by creating multiple virtual environments.

There are already plenty of good posts out there on virtual environments so the aim of this blog post is not to rehash why you should use virtual environments (see here for a good introductory blog post here) or as a quick setup guide (see the Hitchhikers Guide to Python post). It is a quick guide to using virtual environments within Visual Studio. If you have not used virtual environments before it is worth giving these posts a quick read before continuing.

As an aside, Python 3.3 introduced the venv module as an alternative for creating lightweight virtual environments (although the original wrapper pyvenv has already be depreciated in Python 3.6). While this is the correct way going forward, Visual Studio uses the older virtualenv method which is what I am concentrating on here.

Once you have created your Python solution expand it until you get to Python Environments. Right-click on this and choose Add Virtual Environment… from the menu list as shown belowvsve1

 

You can change the name of the folder (defaults to env) which is also used as the name of the virtual environment and the version of Python to use. Click Create to finish and you are ready to go (easy wasn’t it). If you expand the Python Environments node you should see the virtual environment appear.

In the background this has created a folder (the virtual environment) in your working directory with the name given. In case you are unsure, your working directory is the location is the location of the solution which defaults to X:\Users\me\Documents\VS20xx\Projects\Project Name\Solution Name\ – tip, change the default location). This could have been done manually by changing into the working directory and entering the following command (where X:\Python_xx is the installation directory for the version of Python you want to use and env is the name of the folder / virtual environment – if you just want your default version of Python then just pass the name of the folder).

virtualenv -p X:\Python_xx\python.exe env

To install a module into the virtual environment from Visual Studio just right-click on the virtual environment and select Install Python Package… from the menu or if you have a requirements.txt file you can select Install from requirements.txt. If you expand the virtual environment node you will see the modules installed. Once you have all the modules installed you can generate the requirements.txt file from the same menu and it will add the requirements.txt to your project for portability.

What if you want to use this virtual environment from the command line? Inside of the virtual environment is a Scripts directory with a script to make the necessary changes; the trick is to run the correct script from the working directory. The script to run depends upon whether you are running inside a PowerShell console (my recommendation) or from a command prompt. Change into the working directory and type in the following command (where env is the virtual environment folder)

PowerShell: .\env\Scripts\activate.ps1
Command prompt: env\Scripts\activate.bat

The prompt will change to the name of the virtual environment to show activation has succeeded. You can do everything you would normally do from the command line but now you are running against the virtual environment. To confirm the modules installed are only those you have specified type in ‘pip list’ and the version of Python is the one you specified with ‘python -v’.

Update: It appears I’m not the only one to be looking at virtual environments today, see this article if you want a similar introduction but from the command prompt only.

Visual Studio 2017

Visual Studio 2017 has reached the release candidate 2 stage and it is good to see the community version has been retained. I’ve covered the VS 2015 community version in this previous post. Everything from that post remains true in the later version expect who qualifies – open source projects, academic research, training and education remain but only a single professional developer now qualifies, a small team would now have to use the more costly Professional version.

Edit: After I originally wrote this blog post at the end of December MS remove Python from the release candidate without warning. Finally in May it is back and Python appears as a development option next to Node.

The installer for 2017 is different but once installed, the main product retains a similar look and feel. The basic installation does not include the Python tools, not unsurprising given the effort to make VS more modular. Additional support can be added if you go back into the installer; see my comment above and the blog post link for details about what has happened to the Python tools – no longer will a separate download required.

Against upgrading once Microsoft has reinstated Python and it is not longer a release candidate; Python 3.6 is built with VS2015 (as is 3.5) so those compiling modules or embedding Python should stay on the previous version for now. Also if you are already using VS2015 then I’ve not seen anything so would make me want to switch.

It is not an either-or choice as the different versions of Visual Studio will install side-by-side so there is nothing stopping you checking out the latest incarnation like I’ve done.

Microsoft Azure courses

I’ve mentioned Microsoft Virtual Academy before as a great resource for free online courses. As it is a Microsoft site the courses are either on Microsoft products or are presented with a Microsoft slant. There are even a few Python ones on there.

Microsoft has now also launched Microsoft Learning, with Open edX based courses (the same technology used to provide the Mongo University). At present there are only Azure courses on there, we’ll see if Microsoft keep it that way or add additional courses for other technologies over time. If you want to quickly learn about Microsoft’s cloud platform the courses appear clear and concise from bits I’ve done so far.

Windows 10

A bit off topic but now that Windows 10 has been officially released I took advantage of the free upgrade. First impressions; better than 8.1 but not still doesn’t look as good as 7. Initially the start menu appears to be missing the useful menu on the right but these are accessible by right-clicking the start button.

One of the main reasons I’ve been using Windows 8 Pro was to get the integrated Hyper-V feature. After the upgrade I had completely lost the network. Although looking the same as a missing network driver, the cause was that neither TCP-IP v4 and file & print sharing had been enabled. Enabling TCP-IP v4 on the vEthernet adapter solved the connectivity problem. My homegroup is not working so little more digging is needed.

So how is Python. Well IDLE has vanished after the upgrade (a problem that also affects Windows 8 randomly). Other than it works as expected.

Visual Studio and Python

While Idle is good enough to get you started with Python, if you are going beyond the basic scripting having a full IDE is more productive. There are lots of IDE’s available that support Python and it can often be a matter of personal taste but an old timer, Visual Studio, has recently become Python friendly.

While there has been a cut down version of Visual Studio packaged with the express versions for some time, Microsoft released a Community edition starting with Visual Studio 2013. This is the fully professional version aimed at open source projects, academic research, training, education and small professional teams. There is now a Visual Studio 2015 Community edition available for those wanting the latest and greatest.

The Python Tools for VS (v2.1 for 2013 and v2.2 for 2015) have also matured to the point where Python finally feels like a first class citizen in VS. This gives you code complete, ability to select the version of Python to run against (if you have more than one version of Python installed) and all the other features you expect from VS.

Python Tools installs direct from within VS with 2015 so no more hunting around the web for the right version. It comes with some templates to get you up and running (for Bottle, Django and Flask) and also support ironPython.

Where VS really comes into its own is when you are writing in more than one language. I only stumbled across the community edition because I was doing web work. VS supports JavaScript natively and can markup HTML and CSS. Then let you see the results in the browser of your choice using the internal web server in VS. Put all this together and you can write and debug the back end server in Python, modules in C and the client side in HTML, CSS and JavaScript all from with Visual Studio. The code complete alone should justify the hour to two spent learning to use VS.

If you are compiling C modules for Python, note that Python 3.5 was built with Visual Studio 2015, so the community edition gives you everything you need to compile to to embed Python in your C/C++ programs.

For those who are signed up to the Microsoft Virtual Academy, look of for a Python and Django jump start course. Also you can download from Microsoft a Windows 10 VM pre-installed with all the Visual Studio and related SDKs.