Every online search, medical discovery, business transaction, scientific experiment and AI conversation leaves us with a powerful question: What can we learn from the data?
Welcome to the fascinating world of Big Data, Open Data and Smart Data — where everyday observations become useful insights, new opportunities and better decisions.
Big, open and smart are not competing labels. They describe different aspects of data: how much there is, how it can be shared, and how effectively it can be used. One dataset can belong to more than one category.
When data grows so large, varied or fast-moving that ordinary approaches are no longer sufficient, new methods are needed to store, connect, analyse and learn from it.
Think: Millions of medical records, satellite images, online transactions or scientific observations.
Explore Big Data →Data made available for people to access, reuse and share under clearly stated terms. Openness can help researchers, citizens and entrepreneurs build on information that others have collected.
Think: Public transport schedules, weather observations and government statistics.
Explore Open Data →Data made more useful through selection, cleaning, context, connections and analysis. The goal is not merely to collect more data, but to make data fit the question and the decision.
Think: A health dashboard that combines relevant measurements to help a clinician identify a pattern worth investigating.
Explore Smart Data →Big Data is not simply a large spreadsheet. Its real potential comes from finding meaningful patterns across many observations, while recognising uncertainty, limitations and possible bias.
The objective is not to collect data indiscriminately. It is to collect or access the right data, for the right purpose, under appropriate safeguards.
Imagine if every entrepreneur had to collect every fact from scratch, every scientist had to repeat every experiment, and every citizen had to guess how public services worked.
Responsible data sharing can reduce duplicated effort, encourage collaboration and help new ideas emerge.
Open weather observations can help communities, researchers and businesses understand environmental conditions and plan more effectively.
Public transport information can support journey planners, accessibility tools and new services for residents and visitors.
Appropriately shared research data can help scientists validate findings, compare evidence and investigate new questions.
An entrepreneur does not necessarily need to own a satellite, build a weather station or create a national transport database to develop a useful service.
By combining legitimately accessible public data with local knowledge and a well-defined problem, that entrepreneur might create a weather-risk dashboard, a travel-planning application or an environmental monitoring service.
Important: Open does not mean unrestricted. Check licences, attribution requirements, privacy protections, intellectual property rights and permitted uses before reusing a dataset.
Data becomes valuable when we give it meaning. Meaning becomes useful when we learn from it. And knowledge becomes most valuable when we use it with sound judgement.
Raw observations, measurements or recorded events.
Example: 38°C, 39°C, 37°C.
Data organised with enough context to understand what it describes.
Example: A patient's recorded temperatures over three days.
An interpretation informed by evidence, context and experience.
Example: A clinician investigates a possible developing illness.
Using knowledge with judgement, ethics and consideration of consequences.
Example: Choosing an appropriate next step for the patient's circumstances.
Data: Your website records 10,000 visits this month.
Information: Most visits come from mobile devices, but relatively few visitors complete a purchase.
Knowledge: Further analysis suggests that the mobile checkout process may be creating unnecessary friction.
Wisdom: You investigate the problem, improve the experience, protect customer privacy and measure whether the changes actually help.
The numbers alone did not solve the problem. The value emerged through context, investigation, learning and responsible action.
This four-stage progression is a useful teaching framework, not a rigid scientific law. Real-world learning often moves backwards and forwards between stages.
Smart Data is a useful way to describe data that has been made more relevant, reliable and actionable. The term is used differently across industries, but the underlying idea is simple: make data more useful for the decision at hand.
A smaller, accurate, relevant dataset can be more useful than a much larger dataset full of errors, irrelevant observations or missing context.
Data needs descriptions, units, dates, definitions and an understanding of how it was collected. Without context, numbers can mislead.
The purpose is to support a real need: a better experiment, a safer service, a more useful product or a more informed decision.
Powerful computers help AI systems process information. But computing power alone cannot guarantee reliable results. Data quality, suitable methods, evaluation, human oversight and responsible governance matter too.
These foundations work together. Their relative importance varies with the application.
A fluent AI response is not automatically a correct response. Verification remains essential.
Computing power enables processing. Medical information provides context. Suitable models help interpret patterns. Evaluation helps reveal weaknesses. Clinicians and patients supply human judgement, goals and real-world context.
Poorly documented, outdated or unrepresentative data can undermine the system, even when the computers are powerful.
The takeaway: Data is a foundational prerequisite for many AI applications, although not every AI system uses data in the same way. Quality, relevance and responsible use are as important to consider as scale.
Everything
that can have a chip
in it will have a chip in it,
and nearly everything with a chip
in it will become a node in humanity’s ever-growing
big database…
Dr. Hwa A. Lim (林華安博士)
“From biodata to bigdata,” keynote at the 2016 China Bio-medicine Industry Investment & Development Summit, “Fortune 500 (Multinationals) into Jilin,” South Lake Hotel, Changchun, Jilin, China, September 29, 2016.The observation captures a lasting idea: as more objects, services and systems become digitally connected, the opportunity to generate data expands. Turning that data into trustworthy knowledge is the next challenge.
Chips are everyware...
Dr. Hwa A. Lim (林華安博士)
In:The wordplay captures a futuristic tone: AI in the brAIn. Intelligence everyware. Possibilities everywhere——welcome to the future.
Dr. Hwa A. Lim (林華安博士) has presented keynote lectures connecting data, bioinformatics, biotechnology, healthcare and economic opportunity.
His published lecture materials provide a perspective on the evolution from biological data and big data to open data, smart biotechnology and artificial intelligence.
His Silicon Valley connection offers a vantage point from which to consider developments in the technology industry. His lecture history also reflects an interest in translating complex technological developments into ideas that audiences across disciplines can explore.
The following selected lectures illustrate how Dr. Lim connected data with biotechnology, healthcare, scientific discovery and economic development.
The entries below reproduce the lecture details supplied for this page. Where a lecture archive is available, visitors can explore the source materials.
Keynote lecture at the 1st International Conference on Biotechnology and Environmental Safety.
April 14–16, 2009 · National Research Center, Cairo, Egypt.
Keynote at the 2016 China Bio-medicine Industry Investment & Development Summit, in “Fortune 500 (Multinationals) into Jilin.”
September 29, 2016 · South Lake Hotel, Changchun, Jilin, China.
Explore the Big Data lecture archive →Keynote at the 7th International Conference of Veterinary Research, “Development of Animal Wealth: Challenges and Solutions.”
December 5–7, 2017 · National Research Center, Dokki, Cairo, Egypt.
Find lecture slides and related materials →Keynote at the International Conference for Managing the Asian Century, “Linking the Belt & Road with Smart Biotech.”
July 6–7, 2018 · Xiamen, China.
Explore the Big, Open Biodata presentation →Keynote Speaker at the 2nd Golden Gate Life Science Forum.
March 30, 2019 · San Francisco, USA.
Explore Bioinformatics →Chair and Keynote at the Joint Conference: The 9th Annual World Congress on Molecular and Cell Biology (ICMCB-2019) and the 3rd International Conference on Biotechnology.
October 25–27, 2019 · Holiday Inn, Singapore.
Explore related Big Data materials →Across these lectures, the recurring opportunity is to connect data with discovery and practical outcomes. Biological observations become more powerful when they can be interpreted in context, connected with other appropriate datasets and used to investigate meaningful questions.
This is also why data literacy matters beyond science. The same habit of moving from observations to evidence, understanding and sound decisions can help entrepreneurs, students, professionals and researchers.
This gallery introduces Dr. Hwa A. Lim's activities, talks and research-related interests.
Browse the PhotoGPT image directory for additional candidate photographs. Confirm image contents, permissions, accuracy and captions before publishing.
You do not need to become a mathematician, own a supercomputer or memorise a dictionary of technical terms before you can start. Begin with a problem that matters to you and learn the tools needed to investigate it.
Learn to read charts, question statistics, organise information and recognise misleading claims.
Find unmet needs, test business assumptions and discover opportunities hidden in customer or market information.
Help clients organise, interpret and communicate data without necessarily building a large technology platform.
Connect your studies with practical problems and build experience that goes beyond examinations.
Use data to investigate questions, validate findings and uncover relationships across disciplines.
Understand how data, computing and AI methods work together to support useful applications.
Here is a practical learning route for someone with no previous data experience. Adapt it to your goals and the time you have available.
What would you like to understand? Local weather, business costs, transport, sports performance, public health or a topic from your studies?
Look for trustworthy public datasets or create your own small dataset. Check the source, date, definitions, licence and limitations.
Use a spreadsheet to inspect missing values, inconsistent entries, duplicates and unusual observations. Keep track of your changes.
Create a simple chart or calculate a summary. Ask whether the pattern is meaningful and what alternative explanations might exist.
Present your findings in plain language. Separate what the data shows from what you suspect. State limitations and identify what you would investigate next.
Once you are comfortable, explore SQL, Python, data visualisation, statistics, machine learning or a field-specific tool that helps answer your next question.
Download an appropriate public dataset about rainfall in your region. Organise the observations, calculate monthly totals, create a chart and explain one pattern you notice.
Your first project does not need to be complicated. It should demonstrate that you can ask a question, work with evidence, explain your reasoning and acknowledge uncertainty.
New to the subject? Start here.
Big Data describes datasets whose scale, complexity, variety or speed can exceed the practical limits of traditional processing approaches. It becomes useful when we can analyse it to answer meaningful questions.
Big Data describes characteristics of data, such as its size, variety and rate of generation. Open Data describes the conditions under which data can be accessed, reused and shared. A dataset can be big and open, small and open, or big but restricted.
Smart Data generally refers to data that has been made more useful for a particular purpose through selection, quality checks, context, connections or analysis. Because usage varies by industry, always clarify what the term means in a particular project.
Data provides observations. Information organises them in context. Knowledge develops through interpretation, experience and evidence. Wisdom applies knowledge with judgement, ethics and consideration of consequences.
Depending on the system, data may be used to train an AI model, evaluate it, provide context or support its operation. Inaccurate, incomplete, biased or poorly documented data can undermine reliability. Computing power cannot fix every underlying data problem.
No. Many useful Big Data applications rely on databases, statistics, traditional analytics, visualisation or other techniques. AI is one possible tool, not a requirement for every data project.
Yes. You can begin with spreadsheets, basic statistics, charts and small projects using appropriately licensed public datasets. More advanced work may require deeper mathematical, programming or domain-specific knowledge, depending on your goals.
Risks include privacy violations, security breaches, poor data quality, misleading conclusions, bias, inappropriate sharing and misuse. Responsible data work includes suitable safeguards, documentation, access controls and honest communication of limitations.
Data will remain important to AI development and use. Opportunities include better data preparation, integration, evaluation, governance and domain-specific applications. The strongest results require more than scale: they require useful data, suitable computing, sound methods and responsible human oversight.
Big Data does not stand alone. Explore the neighbouring disciplines and the lecture materials that connect data with science, technology and AI.
Explore articles, lecture slides and other materials related to Big Data, biodata, open biodata and the evolution from biodata to big data.
Visit the Big Data knowledge collection →Discover AI-related articles, presentations and videos. Explore how data supports AI and how people can use AI to extend their capabilities.
Explore AI and Intelligence Augmentation →Discover how biological information and computational methods help researchers study genes, organisms, diseases and complex biological systems.
Explore Bioinformatics →Explore Dr. Lim's background and published materials concerning data, bioinformatics, biotechnology and emerging technologies.
Visit Dr. Lim's page →You do not need to know everything before you begin. Start with a question. Find the evidence. Learn from what you discover. Then use that knowledge to create something useful.
Begin Your Data Journey →