Predilexion for Insight

The point: I am proud to invite you to enjoy our new product, Predixion Insight. It is a cloud-based predictive analytics service that can be accessed from Excel 2007 or 2010 (32 or 64 bit) and allows you to  perform advanced analytics on your Excel or PowerPivot data.  To install the Predixion  Insight client, just go to https://www.predixionsoftware.com/products , download Predixion Insight for Excel and subscribe to our Free Beta Trial.

The rest of this post is just me bragging about how cool Predixion Insight is and how one can have fun with it.

  PDXWait

A few years ago, I was working with my adviser on a paper trying to offer biochemists a predictive tool for the bioactivity of HIV-1 protease inhibitors. I am a bit ashamed to admit this, but to this day an “HIV-1 protease inhibitor” is, for me (just like the day I first met one), mainly a vector of 4 numbers in one dataset and 25 numbers in another.

In analyzing those datasets, I used a few different tools, including the Microsoft data mining add-ins. Excel, empowered by the data mining add-ins, was by far the best data playground for me at the time (naturally, no trace of bias here!).  What made this experience memorable is that, after working as a developer on those add-ins for a few years, I was using them as a user and not for a test or demo nor  on a customer’s dataset.

Today, I played a bit with those datasets and tried to replicate that work using Excel 2010, PowerPivot and Predixion Insight. I tried to use the combo as a real user, not as a developer, and see how I can solve a specific problem.

Here are a few things that marked the experience. I will try to blog about each individual feature in more detail in the upcoming weeks, so here is just an enumeration

- Asynchronous task execution. I can launch a task (say, a neural network processing task) and then, while the clouds are crunching the 25-dimensional protease inhibitors, I can schedule additional tasks or use excel. Even better, the processing results are cached in the cloud and available for retrieval when the  job completes, from wherever I log in. Naturally, result sets can be retrieved multiple times

- Power Pivot integration: on one hand, it is nice to know that you can analyze more than 1M rows. However, I found sampling pretty useful in solving this issue, at least for modeling. I found more interesting the fact that I can synthesize very rich datasets using power pivot. Some things are just native there (cross table RELATED calls, aggregations). Others are added by Predixion Insight (outlier and missing values handling, binning of numeric data). All these transformations are wonderfully refreshable and can be nicely applied to new data.

Visual Macros: these are executable reports generated by Predixion Insight for Excel, reports which make repetitive tasks much easier. In brief, after going through a wizard, such as specifying which of 25 input columns should be used in a regression model, a script is generated, script which can be modified and re-executed. Also, multiple such scripts can be chained on the same worksheet and executed as a package. This way , creating 5 regression models on the same 25 columns, with different algorithm parameters, stops being boring. These macros can be used for queries as well and I intend to show some Visual Macros based application worksheets in a future post

Add the fact that I ran the experiments on a tiny laptop running just Office and you’ll see why  the 2010 version of the Excel-based advanced analytics tools is my new preferred data playground. Hence, the point at the beginning of this post. Enjoy!

Technorati Tags: ,,,

Getting Ready For Showtime

Predixion is getting ready for show time. Today is the first day of the Predixion Insight invitational Beta, the first product from our company. This Beta will become public shortly (mid August) so if you are a Data Miner, a PowerPivot pro or  Excel user please keep an eye on www.predixionsoftware.com to get more information. In fact, you should probably check out that page today (if you didn’t recently) as it contains a very crisp description of the Predixion vision for removing the barriers to predictive analytics.

Ever since two years ago when I spent my summer developing a cloud prototype I got hooked on the potential of predictive analytics in the cloud. Predixion Insight is unlocking that potential. I hope and believe that you will find our product a powerful instrument for analyzing your data, anywhere, anytime. We certainly put a lot of work in making it so.

Some information about the product features is already available on Jamie’s blog . I will follow up with more posts describing in detail some product features (after the public Beta becomes available).

A New Must Read Blog (if you are interested in Microsoft BI)

Cristian Petculescu is the Principal Architect of SQL Server Analysis Services. He is in part responsible for everything good that came out of SQL Server Analysis Services in the last 13 years or so. Also, responsible for part of PowerPivot. That really, really fast part. And he decided to blog about his BI endeavors. So, if you want to glimpse into alternative futures for Microsoft BI, check out his blog at http://petcu40.blogspot.com/ .

My first day at Predixion Software

As of today I am working as a Principal Architect (tada!) for Predixion Software, a predictive analytics startup in Redmond, WA. So far, it’s pretty close to the Encarta definition of exhilarate

The company aims to bring predictive analytics technology within reach of every information worker. My past professional incarnation taught me that it may be quite a challenge to make statistics and data mining appealing and intuitive. It also taught me that this is exactly what many users actually expect from an advanced analytics solution, so I jumped at this challenge. (By the way, if you find this kind of problems interesting and appealing then check out our job posting at stackoverflow).

The fact that  Jamie MacLennan is the CTO of this company also helped. I worked for quite a few years with Jamie on Microsoft’s SQL Server Data Mining and I am really looking forward to building other cool products.

 

So – until Predixion has a product I will keep posting about SQL Server Data Mining but also about my trip in the startup world. Meanwhile, if you use SQL Server Data Mining, take this survey  and win a copy of Jamie and Bogdan’s book!

Farewell, Microsoft

Today has been my last day at Microsoft. I spent more than 10 years with the company and loved every bit of these 10 years.  If not for a very promising startup, I would probably be looking forward to spending the next 10 years with Microsoft, in the same BI area that made me write this blog.  My best wishes to the company, the technology and, above all, to the wonderful people that made me love my Microsoft journey!

The next, very-soon-to-show-up-here post will discuss where I am going and what I will be doing there. Talking about future, though, is talking about work, which would spoil this weekend.

This blog will come back to life (although it might move to a different platform). The main topic remains Data Mining and predictive analytics, the core technology remains SQL Server Analysis Services.

 

Regards,

bogdan

 

PS – The Cloud Table Analysis Tools service completed its prototype mission and has been retired. Thanks a lot to all the users! More in my next post

Querying Rules and Itemsets (like the Data Mining Viewers do)

I will try to continue the series started by Jamie, presenting the other set of queries issued by the Microsoft Association Rules viewer. Recently, a question on these queries appeared on the MSDN Data Mining Forums and the poster raised a very good point: while the stored procedures were intended as internal calls for the built-in viewers, external applications and viewers may want to employ them.

So, here is how the rest of the Association Rules viewer works.

Once the viewer is loaded, the first call is something like:

CALL System.Microsoft.AnalysisServices.System.DataMining.AssociationRules.GetStatistics(’Customers’)

The single parameter of this stored procedure is the mining model name.  The result is a one-row table containing the following columns:

Column

Sample Value

Comments

MAX_PAGE_SIZE 2000 The maximum server supported page size for fetching rule and itemsets. This parameter ensures the the viewer will not make requests which will make the server go out of memory, details later.
MIN_SUPPORT 89 Minimum actual support for rules detected by the model
MAX_SUPPORT 2439 Maximum actual support for rules detected by the model
MIN_ITEMSET_SIZE 0 Minimum itemset size
MAX_ITEMSET_SIZE 3 Maximum itemset size
MIN_RULE_PROBABILITY 0.401529636711281 minimum actual rule probability
MAX_RULE_PROBABILITY 0.993975903614458    maximum actual rule probability
MIN_RULE_LIFT 0.514182044237125 minimum actual rule importance
MAX_RULE_LIFT 2.13833283242171 maximum actual rule importance

[Read more →]

Update: Data Mining in the Cloud is back up

To restore your settings:

- in the browser: keep using http://sqlserverdatamining.com/cloud

- in Excel (add-in): go to the Connections button in the Analyze (in the Cloud) ribbon and type the following URL: https://131.107.181.99/CloudDM/TATServices/

With this change, you Excel add-in will once again use HTTPS to secure the data transmission

Thanks for your patience!

Data Mining in the Cloud is temporarily down

On Saturday, November 15th, the connection to the Table Analysis in the Cloud URL is broken. Until the problem is identified and fixed, here are some workarounds:

- For the web interface, use the http://www.sqlserverdatamining.com/cloud URL

- For the Excel add-in, please change the services connection URL. To do that, click the Connections button in the "Analyze (in the Cloud)" ribbon and change the destination URL to

http://131.107.181.101/CloudDM/TATServices/

NOTE: This temporary solution does not support SSL. Your data is transmitted in clear

I’ll post here an update as soon as the servers are up again

Technorati Tags:

Book’s Blog

The “Data Mining with SQL Server 2008″ book  now has a blog. You can check it out at http://www.SqlDataMiningBook.com

Soon enough, there will be some content there:

- various data mining related postings from Jamie and me — all the product related postings on this blog will be replicated on the book’s blog . With a bit of discipline, all the postings will be tagged with the relevant chapter number

- errata (well, hopefully that won’t be the main topic of the blog :-) )

- any new downloads or other information that may be relevant for the readers

Data Mining with SQL Server 2008 + get your own free autographed copy!

DMBook

 

The new version of the SQL Data Mining book is finally available, at least at Amazon. If you are currently SQL DM user, you have an opportunity to get a free autographed copy  by filling out a short survey about the way you use SQL Server Data Mining. For more details about the survey, check Jamie’s blog post.

More details about the book here: Data Mining with SQL Server 2008