As I outlined in my previous blog post, I have recently started including client side (CSS / JavaScript) packages via Bower and Gulp instead of Nuget.
One reader asked me how this approach affected Intellisense for the referenced package. He noticed that after using this approach, Visual Studio was no longer giving him the expected Intellisense for Bootstrap CSS classes.
So I set out to investigate this and sure enough I was also seeing the same problem on my machine. I was not getting any CSS intellisense for the Bootstrap classes. This was especially puzzling because it appears that JavaScript intellisense was in-tact.
Let’s dig into this a little deeper then…
JavaScript Intellisense
Intellisense for JavaScript files is handled by the _references.js file. Visual studio handles this nicely with the example in the article. With Autosync enabled by default, the included files are automatically added to the _references.js file. As such, jQuery intellisense works just as it did when referencing the packages from Nuget. Even though our project only includes a couple combined and minified JS files, everything still works exactly as expected.
CSS Intellisense
So what’s up with CSS? Why isn’t CSS Intellisense working? According to Microsoft, VS 2013 is supposed to inspect any CSS included in your project and make classes from those CSS files available via Intellisense in the HTML editor.
Unfortunately Visual Studio seems to ignore any minified CSS files! This seems very strange to me but at least there is an easy solution.
We can modify our gulp file to also output a non-minified version of the combined CSS file and include that in our project. There is no need to reference this file anywhere. All we need to do is make sure it is included in the project. Suddenly, like magic, everything works again.
Here is an updated version of the Styles task from the Gulp file in my previous post:
// Combine and minify css files and output fonts gulp.task('styles', ['clean-styles', 'bower-restore'], function () { gulp.src(config.boostrapfonts) .pipe(gulp.dest(config.fontsout)); return gulp.src([config.bootstrapcss, config.appcss]) .pipe(concat('app.css')) .pipe(gulp.dest(config.cssout)) .pipe(minifyCSS()) .pipe(concat('app.min.css')) .pipe(gulp.dest(config.cssout));
Update 1 (Feb 23, 2015): Improved code in gulpfile.js
You may have heard that with ASP.NET 5 and Visual Studio 2015, Microsoft will be using Bower as a client-side package manager. That means that packages like jQuery, Bootstrap and Angular will no longer be referenced using NuGet. This might scare you initially, but I very strongly believe that this is a good thing.
Why is this a good thing?
NuGet is a great package manager for .NET libraries and the Microsoft eco-system. The web however, is much larger than Microsoft. The web development world has largely settled on Bower as the defacto package manager for client side libraries. We can’t expect a developer who creates a fancy new JavaScript library to publish their library as a NuGet package. That developer might have nothing to do with the Microsoft ecosystem and expecting them to learn it so we can use their package is just not reasonable. This brings us to the current state of client side packages on Nuget. Many packages are not available on Nuget. Equally as frustrating, some packages are available but are horribly out of date. This seems to be getting worse lately.
So…Given the fact that Microsoft is moving to Bower in the next version of ASP.NET, I have started migrating all my existing ASP.NET projects to use Bower for client side packages.
Here is a step-by-step guide to using Bower with Visual Studio 2013 using the MVC5 File->New Project template as an example.
Once node is installed, we need to install bower using the node package manager (npm). From the command line, run
npm install bower -g
Install Git
Yup..you’re going to need git. Bower uses git to download packages from github. Specifically, you will need to install msysgit and select the “Run Git from the Windows Command Prompt” option.
First up, we will need to create a new bower.json file for our project. This is the file that will contain a list of all the packages your application depends on. The easiest way to do this is to run the bower init command in the your project folder:
The default settings are fine for our purposes. The only setting I changed was marking the project as private to avoid accidentally publishing my project to the public bower registry.
Replacing Nuget Packages with Bower Packages
A new MVC 5 project reference jQuery, jQuery Validation, Bootstrap and Modernizr. Let’s start by adding jQuery using Bower. Bower packages are installed from the command line using the _bower install. _
bower install jquery --save
By default, bower components are installed in the bower_components folder. I don’t include this folder in Visual Studio and I don’t check it in to source control. The contents are easily restored by calling the bower install command with no other options.
Next, we will install the remaining packages. We also need to install RespondJS. This is a responsive design polyfill that is included in the Nuget bootstrap package but is not included in the bower package.
Now, we have all our required client side components installed in the bower_components folder. The bower.json file describes all the client side packages that our project depends on.
Some people will simply include the bower_components folder into the web application and reference the JavaScript / CSS files directly. A typical bower_components folder contains much more than we actually need. I prefer to copy only the files we need to a known location and reference them from there. This is a task that can be easily accomplished using a client side build system like Grunt or Gulp.
Here is the gulpfile.js that I created to roughly mimic the bundles that are created by the MVC new project template. The comments in the file explain what each gulp task is doing.
/// // include plug-ins var gulp = require('gulp'); var concat = require('gulp-concat'); var uglify = require('gulp-uglify'); var del = require('del'); var minifyCSS = require('gulp-minify-css'); var bower = require('gulp-bower'); var sourcemaps = require('gulp-sourcemaps');
var config = { //JavaScript files that will be combined into a jquery bundle jquerysrc: [ 'bower_components/jquery/dist/jquery.min.js', 'bower_components/jquery-validation/dist/jquery.validate.min.js', 'bower_components/jquery-validation-unobtrusive/jquery.validate.unobtrusive.min.js' ], jquerybundle: 'Scripts/jquery-bundle.min.js',
//JavaScript files that will be combined into a Bootstrap bundle bootstrapsrc: [ 'bower_components/bootstrap/dist/js/bootstrap.min.js', 'bower_components/respond-minmax/dest/respond.min.js' ], bootstrapbundle: 'Scripts/bootstrap-bundle.min.js',
// Synchronously delete the output script file(s) gulp.task('clean-vendor-scripts', function () { return del([config.jquerybundle, config.bootstrapbundle, config.modernizrbundle]); });
//Create a jquery bundled file gulp.task('jquery-bundle', ['clean-vendor-scripts', 'bower-restore'], function () { return gulp.src(config.jquerysrc) .pipe(concat('jquery-bundle.min.js')) .pipe(gulp.dest('Scripts')); });
//Create a bootstrap bundled file gulp.task('bootstrap-bundle', ['clean-vendor-scripts', 'bower-restore'], function () { return gulp.src(config.bootstrapsrc) .pipe(sourcemaps.init()) .pipe(concat('bootstrap-bundle.min.js')) .pipe(sourcemaps.write('maps')) .pipe(gulp.dest('Scripts')); });
//Create a modernizr bundled file gulp.task('modernizer', ['clean-vendor-scripts', 'bower-restore'], function () { return gulp.src(config.modernizrsrc) .pipe(sourcemaps.init()) .pipe(uglify()) .pipe(concat('modernizer-min.js')) .pipe(sourcemaps.write('maps')) .pipe(gulp.dest('Scripts')); });
// Combine and the vendor files from bower into bundles (output to the Scripts folder) gulp.task('vendor-scripts', ['jquery-bundle', 'bootstrap-bundle', 'modernizer'], function () {
});
// Synchronously delete the output style files (css / fonts) gulp.task('clean-styles', function () { return del([config.fontsout, config.cssout]); });
gulp.task('fonts', ['clean-styles', 'bower-restore'], function () { return gulp.src(config.boostrapfonts) .pipe(gulp.dest(config.fontsout)); });
// Combine and minify css files and output fonts gulp.task('styles', ['css', 'fonts'], function () {
});
//Restore all bower packages gulp.task('bower-restore', function() { return bower(); });
//Set a default tasks gulp.task('default', ['vendor-scripts', 'styles'], function () {
});
To run the gulp tasks, simply run gulp from the command line. This should output a number of min.js files to the Scripts folder and some css and font files to the Contents/dist folder. In Visual Studio, include these new files into the project.
Removing NuGet Packages
Now that we have all the script and css bundles we need, we can uninstall the original NuGet packages.
To complete the cleanup process, we can delete the App_Start/BundleConfig.cs file and remove the call to RegisterBundles from Global.asax.cs.
Ensuring Gulp runs before running the app
Currently, our solution requires the developer to manually call gulp from the command line to restore bower packages and re-generate the output files.
As we did in my previous blog post, we can use the Task Runner Explorer to ensure these tasks will be run anytime the developer builds the application in Visual Studio.
Open the Task Runner Explorer window (View –> Other Windows –> Task Runner Explorer). Select the default gulp task, right click, select Bindings –> Before Build. Now the default gulp task will run every time the application is built in Visual Studio. This will ensure that the output files are always up to date for the developer.
Conclusion
We now have a fully working application that uses Bower to reference client side libraries and Gulp to define and generate bundles. By using Bower, we get access to a larger and more up-to-date library of client side packages than we did with NuGet. Using Gulp, we have a lot more flexibility in how we define bundles than we did with the ASP.NET Bundling / Minification system.
This approach is more consistent with the way the rest of the world is building web applications today and should make for an easier transition into the next version of ASP.NET.
I am looking forward to a some rare free time over the holidays. I am hoping to watch some training videos and catch up on some blogging (I have a few unfinished posts that I would like to finish).
Updated Sept 16, 2015: Improved sample gulpfile. Updated Aug 30, 2015: Improved sample gulpfile and added sections to answer common questions.
What is Gulp?
Gulp calls itself the streaming build system. No, this isn’t a replacement for build systems like msbuild or nant. In this case, we are talking about building the client side parts of our applications like JavaScript files, StyleSheets (CSS, SASS or LESS) and HTML files.
The basic idea with Gulp is that you use pipes to stream a set of data (usually files) through some kind of processing. As it turns out, it is pretty easy to use and is probably best described using an example.
If you are using VS 2015 and ASP.NET 5, Visual Studio will install npm and gulp for you automatically.
Once node is installed, we need to install gulp using the node package manager (npm). From the command line, run
npm install gulp -g
Setting up your Visual Studio project
Rather than create a new sample project here, I’m going to use the Hot Towel SPA template from John Papa. First, I will create an new empty web application and then install the HotTowel.Angular nuget package.
Install-Package HotTowel.Angular
I wanted to use this template as an example because it is a perfect candidate for Gulp. The application is written using AngularJS and the code is split across 14 different JS files. From a code maintenance / readability standpoint, it is definitely good to split the code into files like this. From an application loading performance standpoint however, loading 14 separate JS files is generally not a great idea.
Here is a snapshot of the traffic captured using Fiddler.
Let’s see what we can do to fix this using Gulp.
Initializing our project for Gulp
First, we need to create a package.json file in the root directory of the your project. We can do this by running npm init on the command line or simply creating a file with the following contents:
{
"name": "YourProjectName",
"version": "1.0.0"
}
Next, we will install a few packages that we will use for this project. Run the following commands from the same folder that you added the package.json file.
Note, the --save-dev option here is telling node to add these packages to a devDependencies section in the package.json file and install the packages in a node_modules folder in the current folder . At any time, you or another developer on your team can re-install all the devDependencies by simply running npm install.
Finally, create a gulpfile.js file in the same folder.
// include plug-ins var gulp = require('gulp'); var concat = require('gulp-concat'); var uglify = require('gulp-uglify'); var del = require('del');
var config = { //Include all js files but exclude any min.js files src: ['app/**/*.js', '!app/**/*.min.js'] }
//delete the output file(s) gulp.task('clean', function () { //del is an async function and not a gulp plugin (just standard nodejs) //It returns a promise, so make sure you return that from this task function // so gulp knows when the delete is complete return del(['app/all.min.js']); });
// Combine and minify all files from the app folder // This tasks depends on the clean task which means gulp will ensure that the // Clean task is completed before running the scripts task. gulp.task('scripts', ['clean'], function () {
Now, if we run gulp watch from the command line, our new watch task will watch for changes to any of our js files. When a change is detected it will trigger the scripts task to execute, regenerating the all.min.js file.
You can learn more about gulp.watch here. Note that at this time, the built in gulp.watch has some bugs that stop it from watching new files. These should be fixed soon but in the mean time you might want to use use the popular gulp-watch plugin.
Integrating with Visual Studio
So far, we have been working primarily in the command line. It would be nice if we could integrate with our existing Visual Studio experience.
Luckily, we can with the new Task Runner Explorer plugin. Once the plugin is installed, open the Task Runner Explorer from the View –> Other Windows –> Task Runner Explorer menu.
This window will show you all the tasks in the gulp file and allow you to bind those tasks to certain Visual Studio events. This way, we don’t need to remember to run the gulp tasks from the command line. The IDE can handle it for us.
What I like to do is bind the watch task to the Solution Open event and bind the scripts task to the Before Build event.
With these bindings, we can make sure that all.min.js is correctly generated when we build the application and also regenerated anytime I make changes to the js files.
The Task Runner Explorer also shows the output of any running tasks. Here you can see the output from the watch task, which is always running in the background in this configuration.
Development vs Production
To simplify debugging, you may want to include all your individual scripts in a development build and only include the single concatenated/minified script in your production build. Take a look at my post on Web Optimization in ASP.NET Core MVC for some ideas on how to this can be accomplished. That post covers ASP.NET Core MVC but a similar approach could be used in MVC 5. With the following in your cshtml file, the individual files would be included when debug=”true” is set in your web.config. If debug=”false”, then only the single concatenated/minified file is included.
Note that this only works for cshtml files in an MVC application as it requires the Razor view engine.
Why?
So, now we’ve seen a very simple example of bundling and minification. As many have pointed out, this could have easily been accomplished with the runtime bundling in MVC5 provided by System.Web.Optimization. I have a few thoughts on this:
Runtime vs. Compile-Time Optimizations
System.Web.Optimization takes the approach of bundling/minifying your assets at runtime. The first time someone asks for a bundle, it will combine and minify all the files in that bundle and cache the results for the next request. While the cost of this is minimal, it has always seemed to me that it is a strange to use server resources to do this task. At the time of publishing our application to the server, we already know what the code is. To me it makes more sense to do this step on the build server or on the developer machine BEFORE publishing the code. Task runners like Gulp take the approach of doing these asset optimization steps at compile/build time.
Note that there are some specific use cases such as CMS tools that require runtime optimizations because the assets might not be known at compile time. For the vast majority of applications, I think the task runner approach is more logical.
Extensibility and Consistency
There is no question that the runtime bundling in MVC 5 provides a better ‘out-of-the-box’ experience. When you create a new project, bundling and minification is setup and working. It is easy to add new files. People generally understand the concepts and don’t need to spend a lot of time fiddling with the bundle configuration. Where System.Web.Optimization starts to fall apart for me is when I want to take things 1 step further.
What if I want to start using a CSS pre-processor like LESS or SASS? There is no way built-in way to tie CSS pre-processors into System.Web.Optimization. Now I need to start looking for VS plugins or extensions to System.Web.Optimization. If we’re lucky, these will work well. In my experience they have some problems, are often out-of-date or are just not available. One big problem with using VS plugins is that I can’t make use of those on the build server. Another problem is trying to make sure that everyone on the team has the right plugins installed.
With Gulp, all I need to do is include a gulp plugin (eg, gulp-less) and add the less compilation step to my stylesheet pipeline. It would be a 1 or 2 line change to my gulp file. The node package manager is able to ensure that everyone on the team has the right gulp plugins installed. Since everything is command line based, it is also very easy to call the same tasks from the build server if necessary.
So the big advantages are extensibility and consistency. System.Web.Optimization is very good at doing a couple things, but it is also limited to doing those couple of things. When we want to take things a little further, we start to run into some pain points with ensuring a consistent development environment. Gulp on the other hand is extremely flexible and extensible in a way that makes it easy to provide consistency for your team.
Wrapping it up
This post really only scratched the surface of what is possible with Gulp. There are 1,000’s of Gulp plugins available for pretty much everything you can imagine. This is not limited to JavaScript files. You could be processing stylesheets, images or audio.
My typical pipeline for scripts in a SPA is TypeScript Lint => SourceMaps => TypeScript Compile => Concat => Uglify. With the magic of a gulp-watch, this all happens anytime I make a change to my typescript file. The only script files I commit to source control are the TypeScript files. The compiled JavaScript and concatenated files do not need to be checked in because they can be generated at any time using Gulp. The build server is responsible for doing that before publishing the application to the server.
You might also be interested in a few follow up blog posts:
I was working on BugTracker.NET today and I started getting this error while trying to use a new package that I had added. The problem was that the library I added was referencing Newtonsoft.Json v4.5 while my application was using v6.0.
Now, this is a common problem that can normally be resolved by adding an assembly binding redirect that forces all assemblies to use the newer version of Newtonsoft.Json that your application is referencing. The solutions are well document over on StackOverflow.
Unfortunately, none of the solutions described on the internets worked for me. I was starting to think I was losing my mind when suddenly I noticed that I was getting 100s of warnings about the format of my web.config.
It seems that the xmlns attribute was causing some problems. I tried removing the offending xmlns attribute and everything started to work as expected. For some reason, this seemingly harmless xmlns attribute was causing .NET to ignore the binding redirects in my web.config.
Today, Microsoft Azure announced the public preview of DocumentDB. Conceptually, DocumentDB is similar to MongoDB, CouchDB, RavenDB and other ‘schema-less’ databases. These are often referred to as Document Databases or NoSQL Databases. I won’t get into the Relational vs NoSQL debate here. Suffice it to say that NoSQL databases have become very popular and are here to stay.
DocumentDB is a PaaS offering on Azure. That means you don’t manage a virtual machine yourself. All you need to do is create a DocumentDB instance in the Azure portal and connect with any of the available drivers/SDKs. Currently, there are .NET, Node.js and Python SDKs available. You can also access DocumentDB directly using the REST API.
The big question is: Does the world need another NoSQL database?
There are clearly some very good options on the market already, so why does DocumentDB need to exist? Well, I think that DocumentDB has a few unique features that can help bring schema-free databases to the huge numbers off developers using relational databases today.
What’s unique about DocumentDB?
There are a ton of interesting features in DocumentDB, and you can read more about them in detail on the Azure website. Here are a few things that stood out for me.
JSON and RESTful
Microsoft tried hard not to reinvent the wheel with DocumentDB. The goal was to use existing protocols, languages, and formats to offer a Document database as a service on Azure.
All functionality including CRUD, query and JavaScript processing is exposed over a RESTful HTTP interface. By offering simple, scalable JavaScript and JSON over HTTP, DocumentDB doesn’t invent in the area of data models, application models or protocols
That’s actually pretty cool! All modern programming languages already have great support for serializing to / deserialzing from JSON and have great support for calling RESTful APIs. This makes DocumentDB very accessible.
Really?!?! Why would we want to do that? Well, the reality is that most existing NoSQL offerings lack support for transactional processing. This is a huge limitation for many development teams that have been able to rely on the transactional processing in relational databases for years.
SQL
Yes, DocumentDB is a NoSQL database, but the reality is that SQL is a convenient way to query data and developers have years of experience with it. DocumentDB provides a method of querying for data using SQL.
LINQ
DocumentDB ships with a LINQ provider capable of handling complex queries and projections. If you are building .NET applications, this will make it very easy query data efficiently from DocumentDB.
What I would like to see
Development Options - Paying $50 per month for a non-production DocumentDB instance is rather steep. I would like to see either a free sandbox area for development or an local emulator like we have for table and blob storage.
Management Tools – If you are used to managing your databases using SQL Server Management Studio, you will probably find the current management / monitoring tools a little lacking. Of course, DocumentDB is only in preview, so I expect this to improve a lot in the near future.
Learn More
Tutorials - Detailed tutorials are available on Azure.
Pricing – Pricing options are somewhat limited during preview. You can expect the pricing to change once DocumentDB moves to general availability. I expect pricing to be very similar to MongoLab for MongoDB.
