SP SIN Questions – Tips on Controlling Performance of SP SIN

As I mentioned in a previous post on why SP SIN is web scoped, Chris Quick asked me a couple of questions over at SPYAM regarding SP SIN and I wanted to elaborate more on the answers.

Chris’ second question is:

2. I don’t see any data caching in the solution. How will this perform on heavily used sites, could this lead to performance issues?

Again, a great question that allows me to show off exactly how flexible and extensible SP SIN is.

First, any code you add to a site will increase performance load on the server. Anything you add from a single line of code manually to a master page to installing a framework like SP SIN will affect performance to some degree.

SP SIN allows you to tailor your performance loads so that you can change the way SP SIN works depending on where you have the resources to spare. In this article, I’ll discuss a few aspects of this from a client-side and server-side perspective.

Client-Side Performance

SP SIN was designed to allow you to tailor exactly what you do and where you do it. To illustrate this, the two out-of-the-box resource types (scripts and style sheets) include a URL filtering mechanism that allows you to load resources only where you actually need them.

This means that you can increase page load time for clients by sending resources like jQuery, style sheets that affect only certain pages, and so on just where those resources are needed. Your bandwidth will thank you, as will your client download speeds.

If URL filtering is not enough for you, however, you can build or download additional resource types that allow you to target and filter even further or maybe in a completely new way. In the 1.0 release (still available on CodePlex) you can see an example of a custom resource type that allows you to send a resource only to site owners, for example. There’s also a YouTube video that shows you how to build custom resource types in SP SIN.

Regardless of what you do in the filtering department, all SP SIN does from a client perspective is tell the client what to download or to behave. This is exactly the same output that you would add manually if you were editing pages or master pages. Anything you normally do to enhance client-side performance, like using minified scripts or dynamic loading frameworks, works exactly like they do without SP SIN.

So, SP SIN does not cost anything for client-side performance, but allows to to increase client-side performance by only loading resources where you need them.

Server-Side Performance

SP SIN runs on the server, so in general, the largest performance impact will happen there. I’m guessing that’s what Chris meant when he asked about performance issues.

The server performance will mostly be impacted by the number of resources you have in your resource list. By default, all resources need to be evaluated to figure out whether they should run, regardless of whether they actually output anything. So, even if you’re on a page that only need to load a single CSS file, but you have 15 resources in your resource list, each of those 15 resources need to check whether they should load for the current request.

The second factor in determining what the performance impact will be is the resource types you use. The resource types run dynamically loaded server code, so if that code is slow or uses lots of resources, then that will adversely affect performance. If you are a resource type developer, I strongly urge you to keep this in mind when you are building your resource types.

Keep in mind that because SP SIN dynamically loads code at run-time, this means that the first time you load a resource type (like after an application pool reset) you will see a performance impact for that request. However, the code remains in memory after loading the first time, so the overhead of the dynamic loading feature goes away for subsequent requests.

So, SP SIN server-side performance depends on how many resources you have and also the performance on the resource type code.

Note: This is another reason why the scope of SP SIN is site rather than site collection; a more narrow focus means less resources need to load for each request.

Regardless of how many resources are in the list or what code runs in resource types, there are still options for controlling performance, however, and this is where the SIN Cycles may be an option.

SIN Cycles?

SIN Cycles were introduced in SP SIN version 1.2, and is a way for developers to interact with the request cycle of SP SIN (called the SIN Cycle). In short, developers can inject code into any stage of the SIN Cycle, and this opens up a lot of opportunities, for example for building performance enhancing solutions.

One way to do this would be to build a simple caching solution. Again, SP SIN uses normal .NET code, which means that there are plenty of software patterns you can use to cache content.

I’m no caching expert, but I built a simple caching engine that would cache output from SP SIN for 10 seconds. The way I did this was by checking in the AfterContextLoad SIN Cycle event whether a cached version existed, and if it did, I would return a false value from BeforeResourceItemShouldLoadEvaluation to prevent the evaluation and execution of individual resource types.

Note: I actually discovered that I had forgotten to implement the handling of BeforeResourceItemShouldLoadEvaluation so I updated SP SIN to version 1.2.1 to fix this. If you’re using 1.2, you can upgrade if you want to use features that depend on this in SIN Cycles, or you can wait for version 1.3 and upgrade then.

I then added around 40 resource items to the list, and fired off a couple of requests to test the impact (I used the code from the SINCycle sample code distributed with SP SIN 1.2 to check performance). Here are the results:

No cache, first hit:

<!–
SP SIN stats:
Start: 3:01:01 AM
End: 3:01:02 AM
Duration: 0.7488048 seconds
Loaded from cache: False
–>

No cache, subsequent hit:

<!–
SP SIN stats:
Start: 3:10:23 AM
End: 3:10:23 AM
Duration: 0.2496096 seconds
Loaded from cache: False
–>

With cache:

<!–
SP SIN stats:
Start: 3:10:41 AM
End: 3:10:41 AM
Duration: 0.1092007 seconds
Loaded from cache: True
–>

As you can see, there’s definitely a benefit to doing caching, even with my limited skills in this area and a 30 minute solution. I’m certain that some brilliant mind who knows more about caching than I do can come up with an even better way of doing caching.

Who knows, you may even want to build a product for this and sell to users :-)

.b

Found this article valuable? Want to show your appreciation? Here are some options:

a) Click on the banners anywhere on the site to visit my blog's sponsors. They are all hand-picked and are selected based on providing great products and services to the SharePoint community.

b) Donate Bitcoins! I love Bitcoins, and you can donate if you'd like by clicking the button below.

c) Spread the word! Below, you should find links to sharing this article on your favorite social media sites. I'm an attention junkie, so sharing is caring in my book!

Pin It

SP SIN Questions – Scope, My Evil Plan, and How You Can Become Filthy Rich!

I continue to receive comments and feedback on SP SIN, and it’s very positive so far. If you have questions, comments, feedback, success stories, bugs, hate, or anything, feel free to let me know.

One set of questions I would like to mention specifically came from Chris Quick on SPYam. With his permission, I’m reposting the questions and answers as blog posts so I can answer more fully and to a larger audience. There are two questions, and thus two blog posts, because I know your attention span is the length of a human hair.

Oh, and to make sure you read until the end, I’ll both reveal my evil plan with SP SIN and tell you how to become filthy rich by the end of this blog post.

Chris Quick in reply to Bjørn Furuknap: I really love SP SIN and have been investigating it for some of the projects I am currently developing. Here are a couple of points that I shared with a co-worker the other day after evaluating the tool. I’d be very interested in your feedback.

1. This is scoped at the web level, so you have to activated it on all sites and subsites in a collection where it will be used. This means to import the scripts, you have to provide them on every site where you intend to use them, correct? This is good because you have more control over where it is running. However, it could become a burden because you have to add the script resources – AND maintain the script resources on multiple sites. Any plans to scope it to an entire collection?

It’s almost eerie how these questions line up with things I’ve already been discussing in several one-on-one sessions over the previous weeks. Because I’ve given these questions considerable thought, I want to elaborate on the answers.

SP SIN Scope

If you’ve started using SP SIN, you’ve probably noticed that the features to activate SP SIN are site scoped (meaning scoped to an individual web). As Chris points out, this means that you have to activate and store the SP SIN resource configuration on each and every sub site in a site collection.

Before I move on to explain why this is the case, I just want to point out that you can store the scripts and resources you reference anywhere you like, including in a site collection level repository like the Site Assets library of the root site if you’re using SharePoint 2010. So, you don’t need to actually copy the scripts or style sheets, you just point your resource to the location of the script or style sheets where they are. Provided your users has access to those files, you won’t need to store them anywhere else.

Now, the short answer to why the SP SIN framework is site scoped is simply that there are no storage mechanisms in a site collection. That’s right, a SharePoint site collection is simply a management and organization thing; it doesn’t actually store anything. Everything you store in a site collection is stored in one of the sites. When we talk about site collection scoped content like master pages, content types, or web part galleries, these are by convention stored in the root web of a site collection, but regardless, it’s still just a site, not a site collection.

The slightly longer answer has several components, one of which, as Chris mentions, is flexibility. By having each sub web host its own configuration, you can target even more focused where you want things to appear. Further, you can delegate responsibility to site owners without having them needing permissions to parent sites or site collection. Finally, although I could have tried to come up with some kind of inheritance system, it would complicate the solution considerably, both for the end users who need to understand where their resources come from, but also for developers who need to design multi-level configurations.

Even if the inconvenience of setting up the configuration on several sites is your most important factor, however, there are still a ton of options available to you.

Let’s say you have something like a project portal where you’ve designed your own site definitions or solutions that automatically set up new sites according to some provisioning mechanism. Well, if you’re already building your solutions, you can easily add a line of two or code to activate SP SIN and a single feature to deploy the configuration. If you’re using a premade or custom made site definition, you can either include SP SIN and a custom feature in those definitions, or staple them on to any site definition (including MySite) using a feature stapler.

You Promised to Make Me Rich!

OK, I can’t actually make you rich, but I have an idea that someone, like yourself, may want to pursue. In fact, it’s part of an evil plan I have to take over the world, get a furry white cat, and send secrets agents to their deaths in pools of sharks.

Note: The second part of my plan is to stop watching Bond movies.

Because SP SIN is both open-source and designed to be extremely scalable using standard SharePoint features only, it’s perfect for someone to extend. And, even though SP SIN is open-source and free, that doesn’t mean your brilliant solution that extends SP SIN must be.

Here’s an idea, one I’ve been turning around in my mind and that might be a good candidate for building a commercial product on: why not build a product that allows people to manage their SP SIN resources either on a site collection scope, or even a farm level? You could do something as simple as adding a site event receiver to automatically activate SP SIN and configuration packs when users create new sites, but you could make it as complex as you’d like.

For example, you could have a fancy interface where the entire site hierarchy was shows as a tree view and a right-click would give you a context menu where you could activate or deactivate SP SIN, install custom resource types, deploy configurations, or even, if you’re very brave, have a button to extract a WSP containing the current configuration for redeployment across test, staging, and production environments.

I’m certain that once SP SIN gains traction in adoption, and once the SP SIN Store comes out to give you an easily accessible marketplace in which to promote your products, making money off building SP SIN extensions will not only be possible but much easier than it is to become rich on Apps. Because SP SIN is licensed under GPL, you can even bundle it up with your product and ship the entire thing to your customers if they don’t have SP SIN already.

So, here’s my challenge to you. Build a useful extension to SP SIN, something that will make life easier for someone. Create a good user story with a problem and explain why your product solves the problem. If you build something good, I’ll feature it on my blog and on the SP SIN CodePlex site. Can I help you in any way to realize your idea? Shoot me an email or a comment, and I’ll get in touch to help in any way I can.

Hm… Perhaps I’ll even run a competition of some sort…

In any case, I have another question to answer from Chris, and maybe even a product idea for you there, so stay tuned.

.b

Pin It

Why SharePoint Needs An Open Marketplace – And What I Intend To Do About It!

Over nine random days the past couple of weeks (SP SIN just reached build 9, which is the RTM build for version 1.2) I’ve been tinkering with SP SIN, a new framework for injecting stuff into SharePoint pages. Which is fine, I’m sure you can read my other blog posts on the topic if you haven’t been paying attention.

One thing that I announced I’m working on is an app store for SP SIN. Initially, it was only intended to allow people to add configuration packs to SP SIN, but then I thought that this might be interesting for other types of solutions too. In fact, why not create a full app store for deploying any type of solution?

SharePoint 2013 introduces the concept of a marketplace, but you may not know that there’s actually been a framework in place for a marketplace in SharePoint 2010 too. The problem is that nobody uses it, and its only contents is a sample package from Office.com, which isn’t useful at all.

So, SharePoint 2013, right? We can now finally get a marketplace where developers can add stuff and you can buy stuff, and everyone should be satisfied.

Here’s why I think the SharePoint Store isn’t going to be green grass and champagne all the way.

SharePoint Store Caveats

Let’s say that you’re a developer building wonderful jQuery-based solutions in SharePoint. Let’s say you’ve built this cool new widget, for example a fancy fly-out menu, a neat way to filter or sort a library, a slide show, something like that. You may have spent a couple of hours on it, nothing you’d like to build a business around, but still something that would be useful not only to yourself but to others.

Let’s say you want to add that to the Microsoft SharePoint 2013 Store. In order to do so, you need to register for an account, which is reasonable, I guess, but then the process of getting your couple-of-hours project listed starts.

Figures for how long this process takes range from ‘just a couple of weeks’ to ‘upwards of two months’ before your cool fly-out menu that would potentially make thousands of SharePoint sites easier to navigate is available. During that time, you’ll need to interact with the process on several occasions, even if your app is as simple and free as copy-pasting a script from a page.

Microsoft wants you to be committed to your App; if you’re just making something useful that you have no plans to support, earn money from, or build a business around, well, you’re still required to go through the hassle.

I can understand this. Quality before quantity. Microsoft wants their App store to be a place where you find only content that has gone through an extensive process to ensure quality. This is great for more extensive Apps and great for consumer security, but not so much for simpler solutions and for those that want more flexibility at the possible cost of the pre-validated solutions in the SharePoint Store.

Another obvious drawback of the SharePoint Store is that it only supports SP2013 Apps. There is no support for SP2007 or SP2010, and there is no support for any other application delivery model, like farm solutions or sandbox solutions. Again, this is Microsoft’s choice, they obviously want people over on the new model and the new version of SharePoint.

The jury is still out on whether the new App model is a great thing, but regardless of whether the market will accept Apps, the undeniable fact is that Apps provide only a subset of functionality. By design, Apps can only do certain things, and this is done to improve security and stability of SharePoint. If you want more power, like what you need to deploy farm solutions, then the SharePoint Store will not offer you any options.

However, this is a problem for vendors, who for the next couple of years at least will still have to support clients on earlier versions of SharePoint. That means that if ACME Corp wants to have both SP2010 and SP2013 clients that want Apps, they need to build and maintain two separate solutions, increasing the development cost, which again trickles down to a higher price for the clients.

WordPress Got It Right!

On reason why WordPress is successful is its abundance of plugins, easily accessible at the click of a few buttons. If you don’t like the way something works in WordPress out-of-the-box, you can bet a fair chunk of money that there will be a plugin to fix your issue. In fact, you’re far more likely to find several plugins solving your problem and you’ll spend time finding out which one solves your problem best. That’s a good thing for consumers because it means they have a choice. If solution A doesn’t work the way you want, then you can pick solution B or solution C instead.

Previously, Microsoft has been explicitly discouraging apps that are similar to existing apps in their various attempts at marketplaces. I haven’t seen any explicit mention of such a limitation for the SharePoint Store (except they say you cannot duplicate your own App), but because the App must go through a Microsoft approval process, they may introduce such a requirement at any time.

Further, WordPress plugins are free, always. That is, the plugins in the WordPress plugin repository are free. A plugin may certainly sell you additional services and there are external sites that sell commercial plugins too, but the built-in stuff is free.

Getting plugins for WordPress is also dirt simple. Search for what you want, click Install, and you’re good to go within seconds. There’s no limitations on the code you can deploy, and the code runs as securely or insecurely as you’ve set up your environment.

One drawback of the WordPress plugin ‘store’ is that there’s a lot of junk there. In fact, finding the really good plugins take time.

Of course, from a vendor’s perspective, because everything in the WordPress store is free, there’s no built-in business model. That may or may not be a problem; after all, if vendors do see the SharePoint Store as a marketing tool in any case, they may not see the big revenue from such a marketplace in any case. Also, regardless of the free model in the WordPress store, there seems to be a thriving community of commercial plugin vendors, many of which have trial or feature limited versions of their commercial offerings available for free.

What Do You Want?

As I mentioned, I’m working on an App store to run on top of the SP SIN framework. I want to make that App store, dubbed the SIN Store, much closer to the WordPress model than the SharePoint Store model. I want to share my ideas with the community and ask for your feedback on the idea, so please, either email me, comment below, or contact me in any other way with your thoughts.

Here are my ideas. None of them are set in stone, so if something doesn’t feel right or needs to change, then I’ll change it.

  1. The SIN Store will have an open submission policy, just like WordPress. I’ll ask application authors to register, but that’s merely to enable them to maintain their apps. If you can upload your solution file and fill out a simple form, you can publish your solutions through the SIN Store, whether they are farm, sandbox, or App solutions.
  2. I will (and already do) support all types of SharePoint applications, not just SharePoint 2013 Apps, and I will (and already do) support all versions of SharePoint. I’ve already demonstrated the SIN Store running on SharePoint 2007, and I’ve shown to several people how the SIN Store now supports sandbox solutions in SharePoint 2010 as well.
  3. The SIN Store is as extensible and customizable as SP SIN is, meaning that vendors or developers can modify it to work any way they want, including adding payment and licensing options. I don’t add any licensing mechanisms out-of-the-box because I know that vendors tend to be rather particular about how they enforce their licensing. However, as a proof of concept for myself, I’ve tested a model that sells a license code through an online payment provider that in turn is installed in SharePoint and grants access to an application. There’s still too much to figure out on that part, though, so no video of just that yet.
  4. The repository of available apps in the SIN Store is modifiable, so companies can easily add their own app catalog, just like you can in SharePoint 2013 (although, again, for all versions of SharePoint and all types of applications). A usage example may be for a hosting provider to add their own catalog of Apps that they can and will support for their hosted clients. It can also be used to block the default SIN Store repository if you don’t like the open submission policy or are freaked out by the possibility of adding solutions that haven’t gone through a rigorous approval process.
    (I’ve also demonstrated this to people in one-on-one sessions. )
  5. The installation process is also extensible so that if future versions of SharePoint introduces yet another model for app development and deployment, then the SIN Store will support that too. I’ve already shown this extensibility in how I support sandbox solution installation and I have the code ready to support SP2013 Apps as well. Even better, it enables users (and developers) to modify how they install their apps, for example to enforce licensing purchases or management (and it’s actually easy too).

There are some thing that I want to change from how WordPress does their plugin store as well. It can be extremely difficult to find the right type of plugin and the only option you have is really to search in the store, hoping that the keywords you use are the right ones. The SIN Store will have a different navigation model that allows you to categorize your applications better. There’s already support for tagging and categorization, and the SIN Store also categorizes content based on platform capabilities (so you don’t see sandbox solutions in SharePoint 2007, for example).

Further, I want to make it easy for developers to request capabilities. In other words, your App may utilize managed metadata which means you need to have a version of SharePoint that supports that, and have that service available. You can, of course, also limit your application to work just on a certain version of SharePoint and not have it appear in the store if that version isn’t available.

Finally, it needs to be incredibly easy for those users who care only about getting the functionality in place, but also flexible enough for more demanding scenarios like scheduled and targeted  deployments. There are already quick-install buttons for all application types and a pending “Configure Installation” button that right now does nothing but is intended to allow users to determine exactly when, where, and how the app installs.

So, based on this, what are your thoughts? Will this be too much and too far? Is there a need for a competing marketplace? Are there features and capabilities I haven’t thought of? If you’re a developer, would you put your app in such a marketplace?

Let me know your thoughts, rants, ideas, and questions…

.b

Pin It