Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
    pexels pavel danilyuk 8112119
    Data Analytics Is Revolutionizing Medical Credentialing
    8 Min Read
    data and seo
    Maximize SEO Success with Powerful Data Analytics Insights
    8 Min Read
    data analytics for trademark registration
    Optimizing Trademark Registration with Data Analytics
    6 Min Read
    data analytics for finding zip codes
    Unlocking Zip Code Insights with Data Analytics
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Where in the World Does All this ESRI World Data Come from?
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > IT > Cloud Computing > Where in the World Does All this ESRI World Data Come from?
Cloud ComputingData MiningData VisualizationDecision ManagementHadoopMarket ResearchPolicy and GovernanceRisk Management

Where in the World Does All this ESRI World Data Come from?

Shawn Gordon
Shawn Gordon
11 Min Read
Image
SHARE

Image

Image

I was left highly impressed, and still deeply curious, about the guts behind what I saw attending the ESRI User Conference in San Diego the week of July 14th, 2014,  For those of you that don’t know, ESRI is the leader in GIS (geographic data, this is the data that makes up a map that Google, Bing, Mapquest, etc., present). You can see my Technorati write up about the show itself, but this column will dig deeper.

The impression I had leaving the conference, was that, essentially, all this GIS and user data magically existed and it was the vendors jobs to come up with interesting applications and innovative ways to present and make that data useful. ESRI was kind enough to set up an interview for me with David Totman, their Public Works, AEC (Architecture, Engineering and Construction), and Survey Industry Manager

More Read

The Joy of Stats: 200 Countries, 200 Years, 4 Minutes
Disease Management Programs – A Time to Revisit
5 Reasons “Data Scientists” are Sexy
Are You in Denial About Governance, Risk, and Compliance?
Examining Big Data’s Potential In Predictive Marketing

Totman ran the survey program in Colorodo Springs. The pipes, the wires, making sure everything was in the right place from a survey perspective. They managed the electric, gas, water and waste water and are one of the largest four utility management groups in the country. Totman has been using the ESRI software since the late 80’s, so he has a great perspective on how the focus has changed over the years.

The commercialization of maps, especially satellite data maps, through companies like Google and Microsoft is a huge impetus behind the growth of awareness and use of maps, but unlike the aforementioned companies maps that really just present satellite data, the data that goes into an ESRI generated map is vastly more rich. For example, on a Bing map, you can zoom down pretty close and get some interesting detail. Going to street view will provide even more data, but again, this data is based on pictures. With an ESRI map and you click on a road, you can potentially see what the speed limit is, the material of the road, are there guardrails, to name a few items.

I’m not doing this in an interview format, there was too much give and take of information, I’m going to summarize down my questions and the information I got.

Q: How is GIS data collected?

ESRI is 45 years old and the GIS industry continues to mature. The rate of that maturation is speeding up tremendously as system become heterogeneous. The late 80’s and early 90’s were the era of data collection. The vendors back then were focused on digitizing the paper maps and breaking it down into data points. So in a city for example, this could be all the streets, traffic signals, fire hydrants, underground pipes, wiring for electrical, cable, phones, landscaping. You name it, and these assets are tracked (or should be).

The hard reality is that a human had to just go and look at things and write them down, usually they’ve had an inventory from the original civil engineering, but that has to all be verified. The job of asset collection is just massive, however, technology to the rescue in the form of LIDAR (Laser Induced Differential Absorption Radar). These can be mounted on a car, plane, helicopter or even just mounted on a tripod and put in a room.

LIDAR on car





The LIDAR device is sending out millions of laser pulses to create a 3D image of the object or region in question, think of it as a 3D scan of an environment. An interesting case study that David told me about was the city of Troutdale Oregon with a population of 17,000 and an area of 6 square miles. Troutdale had recently taken their city asset data and thrown it up on a map and saw an enormous number of problems, like manhole covers on the sides of building, fire hydrants in the middle of a house, fun stuff like that.

City of Troutdale Orgeon


So the city rented a LIDAR car for about $7500 and spent a single day driving around the city and letting the LIDAR system do its work. Once you get back with the data, you can then play the data back and look at the data points. A human can then identify and assign each of those, or you can rent one of the available Artificial Intelligence system to chug through the data and do the work for you, then it just requires a human to QA the results. They will still be some time getting everything cataloged, but it is done now, as long as they track new issues and items, they shouldn’t have to go through this process again.

LIDAR scan results



The upcoming trend is going to be UAV data collection, which takes it from driving a car around, to an Unmanned Aerial Vehicle, which is going to revolutionize the market again. Right now it is not legal in the US because the FAA doesn’t want them running in to commercial aircraft, so the relevant groups are looking in to  how to do it safely. The “hobby” airspace, up to a few hundred feet, is still open, so it is ok to operate there. Privacy concerns and laws are still a big issue and have to be resolved, but think of the possibilities. Rapid data timeliness becomes a reality at this point. Imagine you have a flood zone, you can send out your drone, scans the area, comes back, you upload the data and generate a delta of the changes in the landscape to determine any potential engineering that might be required. You are able to review and evaluate remote issues without putting humans in potential danger just to see what might have happened.  

UAV with LIDAR

Q: How is the data stored?

There is so much data that you can’t just throw a whole city up on to the display.ESRI has been working with “big data” solutions like Hadoop of late on public cloud services like Microsoft Azure and Amazon SE. To have high accuracy though, you have to store a lot of datapoints, all the dimensional and location data. Think about measuring every aspect of your city and what that would add up to. Yea, crazy.

Q: How is the data distributed?

The “ARCGis Online” product has a variety of data that you can get access to. Since the federal government and all the three letter acronym agencies of the government are customers of ESRI, any data that is publicly available, will be available to you. Any entity could decide to make their data public, for example, a city might want to make their road pavement data available to the public. This data could then be access by law enforcement, citizens or even news agencies to make use of for whatever reason. ESRI doesn’t own that data, they are hosting it.

ArcGIS Online.png

Just to be clear, ESRI doesn’t have a data center, they are using Amazon S3 and Microsoft Azure as previously mentioned.

ESRI does have data sets of their own that are available both for free, and for licensing. They have their own demographers who come up with various cool products like “Tapestry”, which shows life groups and their habit patterns, so you could research in there and discover NASCAR fans are good at recycling. Some of their free data is the entire planet of National Geographic maps as part of their base maps. Clients can buy copies of the data to download and use or make use of the cloud copies, it depends on the vendor. A customer won’t typically download imagery however, it is just too voluminous. Speaking of the cloud, the latest mobile version of the ESRI collector app will now allow you to go offline to work on and collect data and then when you get back online it will autosync everything back up to the cloud.

A couple other interesting products that ESRI offers, are, “Community Analyst” and “Business Analyst”. With Community Analyst you could use this for social services or religious types of organizations. A thought would be population density versus gender and age to look for trends or groupings for support or activities.

With Business Analyst, you can look into various trends to determine the placement of a business for optimum return. I ran into a guy at the ESRI user conference who was explaining how they used this to determine where to put new Walgreens stores. I have always wondered how they make these decisions, especially when you see 3 Starbucks within 2 miles of each other.

I was surprised and glad to find out that both the city I live in and my church own ArcGIS, which means I might be able to convince them both to make better and more interesting use of the tools available. I hope this overview gives you some sense of what is possible with the software suite from ESRI, it might be that your organization already has it and isn’t using it.

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

langgraph and genai
LangGraph Orchestrator Agents: Streamlining AI Workflow Automation
Artificial Intelligence Exclusive
ai fitness app
Will AI Replace Personal Trainers? A Data-Driven Look at the Future of Fitness Careers
Artificial Intelligence Big Data Exclusive
crypto marketing
How a Crypto Marketing Agency Can Use AI to Create Powerful Native Advertising Strategies
Blockchain Exclusive Marketing
data driven insights
How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
Analytics Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

Stuck in First Gear

5 Min Read

DIALOG Sodexo – Workforce Management

5 Min Read

SAP & IBM Partner to Dominate Enterprise Cloud

2 Min Read

A Farewell to Guns

3 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots
ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?