'Net Features : big data

Conversational Marketing Requires Big Data

Pete Prestipino — Tue, 22 Jan 2013 18:55:00 GMT

There is more pressure than ever for digital marketing departments to measure performance and optimize the total marketing mix. To do that you'll need access to some robust analytics of course; fortunately, there's no shortage of solutions to make this digital dream a reality.

Case in point, Neolane, which provides a conversational marketing platform, has released a new version of its marketing analytics product which it hopes will do just that. And it's doing it all with access to big (and better) data.

While there is no shortage of marketing paltforms which provide some level of data, what may make Neolane's updated solution potentially appealing is that it aims to empower marketers to optimize, excute and monitor their own campaigns without requiring a "Data Scientist or employing additional help from IT departments;" something which you're probably all too familiar with.

“With Neolane Marketing Analytics, we are offering unique data analysis features for marketers to address the emergence of Big Data and have a better understanding of their customers’ behavior,” stated Stephane Dehoche, president and CEO, Neolane.

“Most of our clients have dramatically increased their productivity and are now measuring their customers’ reactions in real-time. Marketers are therefore empowered to effectively use data and deliver personalized cross-channel experiences.”

Analytics - The Hiring Trend to Watch in 2013

Pete Prestipino — Mon, 14 Jan 2013 20:00:00 GMT

One hiring priority that has quickly emerged over the past year is that of the data analyst.

According to a recently released Dice.com Hiring Survey, those familiar with analytics are the fourth largest talent pool (within the tech segment) that recruiters and hiring managers were interested in (preceded by Java/J2EE Developers, Mobile Developers, and .NET Developers and followed by Software developers).

But where do you go to get the education and training required to become a big data analytics master? There is currently a rather sizable gap in big data talent, but colleges and universities are indeed offering programs now – from Northwestern to Drexel, MIT and Stanford – so there likely won’t be a shortage for long.

Ideally, any program chosen will address data holistically and within a business context, including predictive (forecasting), descriptive (business intelligence and data mining), and prescriptive (optimization and simulation) data analysis.

Ex-Mobile Exec Tired of Reports, Starts IDInteract

Amberly Dressler — Tue, 11 Dec 2012 20:15:00 GMT

When IDInteract Founder, CEO and Chief Architect Matt Standish managed 1,769 applications, 25 petabytes of data and seven channels, as well as spent hundreds of millions of dollars to do so as a mobile executive at companies like AT&T, T-Mobile, Motorala, what did he get? A ton of reports.

Despite the countless resources at Standish’s disposal, he wasn’t able to take unstructured data and make it actionable. Using that data in a very relevant way to understand customer intent is one of the many reasons he founded IDInteract in January 2012, which he says was the right company at the right time.

Eleven months later, IDInteract launched its Demand Exchange platform, which integrates raw data from social sources, such as Facebook and Twitter with client mobile applications and proprietary CRM systems to help marketers identify and engage with highly qualified individuals and prospect clusters.

With predictive analytics and propensity-to-buy algorithms, Demand Exchange identifies consumers’ intent to make a purchase in real-time, enabling marketers to reach out with targeted offers via Facebook posts, direct messages, and personal tweets designed to meet individual demand.

“To fully enable digital marketing we need to shine a brighter light on customers by leveraging the ‘dark’ unused data from their social and mobile activity in addition to any useful CRM data,” said Standish. “Getting at dark data will help enterprises really understand what their customers and prospects need most, and satisfy that demand. The problem is how to collect, analyze and leverage all that data so it’s truly useful for driving meaningful customer interactions leading to revenue. With the launch of Demand Exchange, IDInteract ups the ante, flipping traditional supply-focused CRM on its head with a fully integrated offering leveraging multi-channel data to monetize consumer demand.”

One way Demand Exchange does this, is by providing businesses customer profiles. These profiles include the likelihood a customer will be a first-time purchaser or their potential for usell/cross-sell and win-back based on demand signals. Profiles also include the consumer’s privacy on Facebook, Twitter and Google+ (e.g. high, medium or low), as well as their persona mood (e.g. content). Additionally, businesses can see the person’s last social media updates, mobile phone number, number of mobile downloads and the list goes on.

Key features of the Demand Exchange platform include:

IDIntent – Sources data from social media channels, client mobile applications and CRM systems. By sourcing “dark” data from social media and mobile applications, Demand Exchange enables clients to tap into largely unused but highly valuable real-time information about what consumers are doing, considering and purchasing. This channel agnostic approach to data collection eliminates the need to invest in channel specific tools, and paints a more accurate picture of consumer demand for particular goods and services.
IDPersona – Consolidates multiple data points into singular contextual consumer views for individuals, groups, events and communities called “Personas.” Personas contain and continually update privacy, viral score, brand consideration, and historical and potential revenue contribution information. Consumer “demand signals” within Personas are analyzed using proprietary algorithms to determine real-time intent. Personas allow a marketer to sense shifts in market perception and understand the real-time demand curve for their company and its offerings.

IDAnalytics – Predictive analytics and propensity to buy algorithms are applied to Personas to determine the highest revenue potential prospects for each client. Using Visual Market Intelligence (the abstraction of raw data into easy to understand graphic formats) the Demand Exchange user interface presents this information to clients marketers can quickly focus on the most valuable prospects relative to their business objectives, leading to increased revenue and ROI.
IDOffer – Empowers marketers to manage a standing inventory of offers ranging from reference information, content links and promotions. IDOffer automatically recommends prospects for client interaction pre-aligned with available offers. This automation capability introduces efficiency to what is currently a labor-intensive and manual process.
Measurement & ROI – Continually measures, tracks and reports the status of customer interactions to help enterprise digital marketers measure campaign effectiveness and ROI.

IDInteract is working with a number of large companies, such as big retail stores, to pro sports teams, to subscription services. The company’s primary focus is large organizations, however, it does have a very automated system that small- and medium-sized businesses could get onboard with.

Eight Tips for Working with Big Data

Administrator — Fri, 16 Nov 2012 16:15:00 GMT

By Abigail Lefkowitz, Data Analyst at MaassMedia, LLC

When describing a set of data, the word "big" doesn't just refer to size. "Big" implies that the data comes from multiple sources and is too complex overall for conventional business intelligence (BI) software to process in a practical amount of time and with sufficient computing memory.

It has been shown that the amount of data in the world is growing at a rate of 50 percent each year and more than doubling every two years. Since data volumes are not getting any smaller, it is imperative that organizations attempting to gain knowledge and insights from their data establish a methodology for storing, analyzing, and interpreting big data.

When you are ready to delve into your data, keep these tips in mind:

1. Start small, think big

Sampling is widely accepted as a preferred standard practice. Sampling takes a random, unbiased selection of individuals from a population to estimate the characteristics of that population. A statistically significant sample should produce results similar to that of the entire population.

First, choose the appropriate sample size to ensure statistical significance. Stated simply, a result is statistically significant if it is unlikely to happen by chance. Using multiple, but independent samples of the same size will increase the statistical significance. Next, randomly select that number of data points from the population and carry out the analysis as planned.

2. Use a unique identifier to connect different sources of data

It is advisable to link multiple data sources using some common denominator in order to create a more complete picture of visitor activity and set the stage for more in-depth, transformative insights. To connect data from various sources, each source must assign a unique ID common to every customer or visitor. Web analytics platforms, such as Google Analytics and Adobe SiteCatalyst allow unique IDs to pass through visitor cookies that are assigned when someone visits a website. These IDs can be linked to any other source of data that captures the same unique ID.

For example, a website utilizing a Web-based survey can assign each survey respondent a unique ID that gets passed from the site into your Web analytics platform. As a result, responses to the survey can be connected to behavioral Web data such as page views, time on site, etc. Without having both the unique survey ID and visitor ID, these types of merges would not be possible.

3. ETL - Extract, transform and load

To obtain your data, you will need to extract it from each source and store it in a place with sufficient space, such as an in-house FTP server or data warehouse. Unstructured data needs to be transformed into a consistent format that allows for data analysis. Removing duplicates, grouping values and concatenating variables are a few examples of possible transformations. When performing these manipulations on millions of rows of data, using structured query language (SQL) can cut down on both processing and overall analysis time.

After extracting and transforming the data, it must be loaded into a storage location or data warehouse. It is recommended that the data be formatted prior to being loaded into the data warehouse for easier access in the future.

4. Select appropriate business intelligence tools

Standard software, like Microsoft Excel, does not have the capacity to manipulate big data and carry out simple calculations in a timely manner because of the data's volume and complex nature, coupled with the fact that it is commonly stored in separate silos with limited accessibility. Microsoft Access is an example of a powerful database that allows you to join and link datasets, especially when connecting unique IDs from different sources.

You will also need two additional analysis tools—one that can perform in-depth analysis and one that can produce visually appealing graphs and charts. It is difficult to find a program that can tackle both effectively. SAS and SPSS/IBM are two examples of statistical analysis programs with excellent data mining capabilities, while Tableau is a tool that produces attractive and dynamic images.

5. Incorporate demographic data

Overlaying demographic information with your dataset is helpful and highly recommended to better understand who your target audience(s) might be. FICO and Experian collect a variety of consumer data scores based on credit card transactions and demographic status. For example, FICO can help you predict whether you will be taking your medication every day based on certain variables such as length of time living in one location and number of years owning a particular car. Again, pending a unique ID, the demographic information can be linked to the existing dataset.

6. Hire a "data scientist"

Making sense of big data can be daunting. Instead of taking it on alone, you might be better off hiring experienced analysts who are comfortable working with massive sets of data. Employing one or more people who can dedicate their time to analysis will allow for smarter and more accurate, meaningful and transformative findings in a shorter amount of time. Look for analysts who are knowledgeable about storage, manipulation, importing and exporting, merging, sub-setting, hypothesis testing and mining for insights.

7. If possible, automate reports

When working with data that is continuously streaming in, automated reporting can save both time and money. Instead of manually reformatting the data each time a new chunk comes in, set up an automatic report with a filter that captures only the necessary variables and manipulates them in such a way that the data is more streamlined to the end goal.

Reports must first be customized to the preference of the end user, and then they can be automatically delivered to one or more recipients at regular intervals. Adobe SiteCatalyst and Google Analytics are two examples of Qeb analytics platforms that can automate daily, weekly and monthly reports.

8. 75 percent investigation, 25 percent discovery

As with any type of analysis, it is a good rule of thumb to spend approximately 75 percent of your time testing pre-developed hypotheses and the other 25 percent poking around in the data with no specific direction in the hopes of uncovering nuggets of knowledge. It is likely that at least one hypothesis you have formulated will lead to additional analytical questions about the scope of the original hypothesis. Therefore, the majority of the time spent testing hypotheses will inevitably lead to hidden findings. Devote a smaller fraction of your time slicing and dicing the data in unexpected segments because many times buried insights can be found when looking at familiar information in an unfamiliar way.

Political Campaigns & Digital Marketing: Did They Miss the Mark?

Administrator — Mon, 05 Nov 2012 17:30:00 GMT

We all know how online marketing and social media can be a goldmine of potential consumers.

Brands like Whole Foods Market and Samsung have succeeded on Twitter with as many as 3 million followers each. Coke triumphs over Pepsi on YouTube with about 108 million views and 83,700 subscribers. Among media brands, Disney dominates with nearly 38,700,000 Facebook fans. In 2011, Netflix’s display advertising received 36.9 billion ad impressions and was cited as one of the top 10 display advertisers in the United States.

But when you look at political spending, campaigns are still spending more on traditional media platforms than online. Consumer brands are learning how to harness online and social media, but political marketers can take a lesson from those finding success with digital marketing. Online and social media is an advantageous investment if political campaigns wisely use online channels, especially their analytics tools.

But if online and social media are so powerful then why are the presidential candidates and Super PACs not investing more into online advertising? Why are campaigns and Super PACs less interested in social media than offline?

Traditional media is still the force in political spending. True, online spending is six times higher for this year’s election than it was in 2008, but for political advertising, TV is the main medium—approximately $7.3 billion dollars for broadcast TV, cable and radio versus $159 million for online, according to Borrell Associates.

Historically, political campaigners had two rivals in the final stretch on an election: a Democrat versus Republican, and those two camps spent money on their campaigns. Now, Super PACs have flooded the ad market, driving up the competition and the inventory price within paid search advertising. For instance, if there are only two people bidding on keywords, there’s a limited competition with keyword bidding. However, if you have 10 people bidding on the same keywords, it drives up the price of inventory and increases the cost-per-click.

Obama advisor, David Axelrod, may have put it best when he said that local TV is “still the nuclear weapon” within an election. This doesn’t mean, however, that political marketers are not concerned about the digital battleground. One of the biggest wins is the investment by the Obama campaign in its social and online media. By numbers alone, Obama leads in many fronts on the digital battleground—from Twitter, online display ads and paid search marketing. But you need more than mere numbers to be successful. Candidates, like consumer brands, can better leverage online and social media to keep and win supporters.

First, test, track and analyze your political messaging in the digital space. For instance, let’s say you have a rally in the battleground state of Ohio and want to ensure high voter turnout. Social and online marketing are useful to engage and motivate your base, as well as communicating with your base quickly and in real-time. As you gather your online following, you can test and track which messages resonate with those people before creating a costly TV ad or making a speech. That’s a competitive strategy that marketers can use before going to market.

Second, marketers should invest marketing dollars into online display ads to target undecided voters. Marketing is not solely about the size of your budget but who’s going to spend their budget more intelligently. Display advertising or banner ads can be the tipping point because of its granular level of targeting. Within this channel, you can find such information like demographics, geography, and user behavior, making display ads far superior than TV and allowing marketers to gain more precise and targeted reach.

Third, think like your competitor both online and offline. Some people believe that “big data” can solve many marketing problems but that’s a misconception. Like any amount of information, you need to gather, organize and properly analyze it. You need the attention to detail and objectivity to find any missed opportunities or big wins – for you and your competitor. By investing more resources into data analytics tools (Google Analytics and Adobe SiteCatalyst), you have greater insight into the effectiveness of your online advertising than with other medium, such as how many donations are you receiving from a banner ad versus a paid search campaign.

To sum it up, it might not be that political campaigns are less interested in digital marketing. They also need to consider if they are investing their ad dollars where it counts—through quality analytics.

About the Author: Jon Morris is the founder and CEO of Rise Interactive, a digital marketing agency in Chicago, specializing in digital media and web analytics.

The Slow Adoption of Big Data

Pete Prestipino — Mon, 22 Oct 2012 18:50:00 GMT

Despite all the hype and potential, most companies are essentially stalled in the preparation phase of their big data efforts.

A new study from OpTier (Making Big Data Analytics Fast & Easy) reveals that few enterprises are able to apply any level of useful analytics to big data for competitive advantage because of the time and costs associated with traditional approaches. But it needn't be that way. According to OpTier President Andy Wild, what's missing today from Big Data analytics is context.

"Today's leading companies are struggling to take advantage of the volume and variety of data available within the four walls of the enterprise” said Wild. “By rethinking the way they analyze Big Data and capturing transactions that are already in context, CIOs can fundamentally change the economics and process of analyzing Big Data - saving 50 percent of time spent and cost."

The OpTier study revealed that companies are not gaining business advantages from their big data because it is distributed across multiple silos and lacks the content and uniformity necessary to allow analysts to leverage it quickly. OpTier’s research indicates that when companies can apply context to their data however (e.g. understanding the interrelationships between various data sets), things (good things) begin to happen particularly when you have an efficient process in place.

Leverage Big Data Better with Automatic Segmentation

Michael Garrity — Thu, 20 Sep 2012 20:45:00 GMT

We all know that it’s difficult to manage big data, which is, by its very nature, huge and disorganized. But, as those companies that have found a way to make use of it can attest to, big data can also offer some incredibly valuable insights about your customers that can greatly benefit your Web business, if they’re leveraged correctly.

One of the simplest methods that many have found for making sense of big data is to segment it into different groups to make it easier to decipher, understand and use. This is likely why big data analytics provider Medio Systems just announced a new module for its inGenius Suite called Clustomers that provides automated audience segmentation to help increase customer engagement and monetization capabilities.

The inGenius Suite of predictive analytics solutions was built to help Medio customers gather insights about their data, and then use that information to deliver relevant and actionable content or services to their own customers. With Clustomers, organizations will be able to automatically surface important results quickly and easily, and use that information to identify the most significant factors on their sites that provoke user retention and engagement in Web or mobile applications.

Clustomers uses activity and user data that has been acquired through either Medio’s software development kits or its Data Collection Service. The module takes the information and then analyzes attributes or features that are associated with key user segments; after that, it summarizes the correlations that can provide the most value to the customer’s business. This allows users to quickly pull out actionable (and meaningful) connections about their customers without laboring through a bunch of reports.

What really makes Clustomers a useful module is that it takes data that is more difficult to obtain, such as inter-session frequency, average session length or mid-application abandonment, and analytically determines which features are the most statistically significant, thus, eliminating the reliance and cost of data analysts required to filter out findings that appear related simply by chance. This allows Medio customers to turn data into action with haste.

According to CEO Rob Lilleness, “These deep insights give our customers the ability to quickly make changes to their customer experience and acquisition efforts that will drive more effective monetization.”

Key Interest Drivers in Big Data Analytics

Pete Prestipino — Mon, 10 Sep 2012 19:51:00 GMT

Big data is (very) big business but what’s driving the trend? After a stellar 340 percent increase over its own Q1 sales numbers, big data analytics provider Pentaho is offering up some insights.

According to Pentaho’s own data, a full 70 percent of its new customers are deploying on Hadoop, with the remaining new customers split evenly between NoSQL databases (MongoDB and Cassandra) and analytic databases (e.g. Greenplum and Veretica).

Pentaho played its hand in the big data game very well, releasing some of its technology to the open source community under the Apache License. That likely resulted in much more adoption in the broader developer community and brought the number of downloads for Pentaho’s big data technology to more than 65,000 over the last few months.

So what else is driving interest in big data analytics?

New Uses: Pentaho also saw a “growing spectrum” of companies getting involved with big data – spanning industries including digital media, mobile apps, gaming, healthcare, finance and more - and creating analytics products to understand customer behavior, lead conversion, security threat patterns and even supply chain optimization.

More Tools: One of the obstacles in big data analytics optimization has been the availability of tools which simplify complex scripting have emerged too, including Pentaho’s own visual design studio which creates MapReduce jobs without coding and provides an engine to execute them in Hadoop.

The Future Looks Bright for Hadoop MapReduce

Michael Garrity — Sat, 04 Aug 2012 16:00:00 GMT

The big data boom has changed a lot on the Web. Most notably, it has spurred the creation and development of many new technologies, and none of them has made a bigger impact than the Apache Hadoop open source software framework.

As the Web continues to grow and we develop more ways for users to access it, big data will only get bigger. Because of this, Hadoop is one of the most important projects out there. But you don’t have to take my word for it, just look at a new report from MarketAnalysis.com to get a look at the exponential growth of Hadoop MapReduce.

According to the “Hadoop-MapReduce Market Forecast 2013-2018,” Hadoop MapReduce is expected to grow at a compound annual growth rate (CAGR) of 58 percent by 2018, which accounts for approximately $2.2 billion.

It seems clear that Hadoop is well-positioned to become the industry standard technology for managing big data and business intelligence solutions, and it presents opportunities for business environments that rely heavily on big data. As they begin to see a growing need to accommodate increasing amounts of data that they have to process, store, and analyze, they also see more cost-prohibitive pricing models being imposed by established IT vendors.

These issues negatively impact the IT budgets of many companies, but thanks to the open source nature of Hadoop, it becomes more of a viable and cost-effective solution every day.

To purchase the full report, click here.

Big Data Management Producing Big Returns

Linc Wonham — Tue, 05 Jun 2012 15:30:00 GMT

“Big Data” has been one of the biggest buzzwords in the online business world the past few years, but just how big are the returns being delivered to companies that are successfully leveraging big data management?

Pretty big, according to one new study. Business technology solutions provider Avanade surveyed more than 500 business executives and IT leaders and found that most of the investments that companies are making to manage big data are paying off in spades.

Eighty-four percent of respondents believe that big data helps them make better business decisions, and 73 percent of companies have already used data to increase revenue by growing existing revenue streams (57 percent) or creating entirely new sources of revenue (43 percent).

Beyond tangible business value, evidence shows that big data has become pervasive. More employees in businesses have greater access to increased technology options for managing and analyzing data. The majority of companies surveyed (57 percent) noted that in just the last 12 months, more technology options became available to help analyze and manage data.

More importantly, research shows that companies are investing in this technology, with almost every company surveyed (91 percent) reporting the use of tools to manage and analyze data today.

“Big data has gained a top spot on the agenda of business leaders for the real value it has begun to create,” says Tyson Hartman, global CTO and corporate vice president of Avanade. “Today, the technologies and skills used to leverage big data for business purposes have reached a tipping point – new types of data supported by better tools to leverage it enable companies to find financial and competitive benefits from their data.”

It’s Become Everyone’s Business
Big data has moved beyond the walls of IT. Today, 95 percent of businesses do not consider data analysts a part of their IT staff. Instead, companies are now distributing that expertise to line-of-business groups throughout the company.

The majority of respondents (58 percent) report that data management is now embedded throughout their business as a dedicated function. Further, over half of global companies (59 percent) say more employees than ever before are involved in making decisions as a result of more widely available company data.

Still Work to be Done
However, as with many major technology trends, the benefits of big data come with meaningful challenges. Eighty-five percent of respondents still report obstacles in managing and analyzing data, including being overwhelmed by sheer volume to data security concerns to not having enough dedicated staff to analyze the data.

The majority of stakeholders (63 percent) also feel their companies need to develop new skills to turn data into business insights.

Findings also show that three other major trends – employee mobility, cloud computing and social networking – are making the data deluge more challenging. The majority of respondents reported that employee mobility (73 percent), cloud computing (65 percent) and social networking (61 percent) are all causing their company to rethink its data management strategy.

“The challenges of big data remain, but the opportunities are even greater,” says Hartman. “Business leaders are moving from defense to offense in their data management strategies. Forward-looking companies are empowering more people across the enterprise with the tools and skills needed to make better business decisions and ultimately, harness the power that big data promises.”

Biz Intelligence Trends Down Under

Pete Prestipino — Wed, 15 Feb 2012 17:30:00 GMT

Revenue from business intelligence software – including BI platforms, analytic applications and corporate performance management software – in Australia is forecast to grow over 11 percent to 406.4 million in 2012 according to Gartner.

Revenue from BI software worldwide, according to Gartner, is forecast to reach 12.5 billion in 2012, up 7 percent over the prior year. The opportunity in the Australian market and the BI market worldwide comes on the heels of the dramatic increase in big data (large datasets, which, enables marketers to better identify subtle patterns and trends). The challenge comes obviously in handling big data and accurately analyzing it.

“The extreme volume, velocity and variety of data will add stress to current approaches to BI and are increasingly capturing the attention of both the BI industry and buyers,” said Gartner managing vice president Ian Bertram.

The significant challenges of big data in the BI market could deepen a widening BI maturity gap between organizations in Australia according to Bertram. “In Australia, many organizations are still focused on foundational BI concepts that revolve around strategy, skills, competencies and process,” he said. “Then we have organizations looking for more advanced analytics capabilities to build on those foundations. This is really where they need to be to set themselves up for the future.”

Related Reading from Website Magazine:
- Domain Name Registration Down Under

Linux Adoption Trends - Cloud Computing, Big Data

Pete Prestipino — Mon, 23 Jan 2012 21:05:00 GMT

Enterprises will add more Linux to support their cloud computing and big data initiatives according to the “Linux Adoption Trends 2012” report from the Linux Foundation.

Eight out of ten survey respondents indicate that they have both added Linux servers in the last 12 months and plan to add more in the coming year. The growth of Linux comes at the expense of Windows servers – only 21.7 percent of respondents indicated they are planning an increase of Windows servers in the next five years.

Additional highlights from the report include:

- More than 75 percent of respondents expressed concern about supporting “Big Data” and 72 percent are choosing Linux to support it.

- Fewer issues are impeding adoption of the operating system, with technical issues cited by respondents dropping 40% from the last report.