Data Sources, Quality and Ethics

Start with the question you need to answer. Then ask who collected each possible source and how closely it describes the people, places, and period you need.

Follow the data to its collector

Your sales records, website logs, and support tickets are first-party data. An airline's booking records shared directly with a hotel partner are second-party data for the hotel. A broker's combined audience file is third-party data for its buyer. These labels describe the relationship to the collector; they do not rank every file's quality.

Keep the other distinctions separate

Internal data is held inside your organization; external data comes from outside. Primary data is collected for your current question; secondary data is reused from another purpose or time. Old sales records reused to investigate a new pricing question are first-party and secondary. A new survey designed for that question is primary.

Open data is accessible and reusable by anyone for any purpose, including commercial use, under an open license. Statistics offices, city governments, and weather services publish examples. Read the license: attribution can require credit, and share-alike can require adapted versions to use the same license. A public file that bans commercial reuse is not open data in this strict sense.

Work through a source choice

A bike-sharing company asks why weekend rides fell 15% this spring.

1. Its trip records show which stations and times lost rides. They are first-party evidence about its own service.
2. Transit ridership shared directly by the city agency adds second-party evidence about nearby bus and train use in the same weeks.
3. Open daily rainfall and temperature records help investigate weather as a possible contributor.
4. A broker's neighborhood file adds demographic context, after its dates, methods, and permissions are checked.

These sources support different parts of the investigation. A fall in bike rides alongside a rise in train rides does not establish that the same people switched services.

Check the channel and the detail

Surveys reach respondents; interviews capture what people say; observations record what someone notices. Sensors cover installed locations. Website logs cover actions on that website. An API lets a program request data from another system; web scraping copies content from web pages. Neither channel guarantees completeness or permission to reuse the result.

Check granularity, meaning the detail of each record. Monthly city totals cannot answer an hourly station question. Compare sources for coverage, documentation, allowed uses, acquisition effort, and cost. A huge population file may be less useful than a small, well-documented file about your own customers.

Data structure describes how information is arranged. It tells you what can go directly into a calculation and what first needs interpretation.

Recognize three forms

A relational table gives each column one role. That makes sorting, filtering, counting, and grouping straightforward. Semi-structured records can have optional fields or lists: one customer may have two phone numbers and another none. Unstructured content needs reading, listening, viewing, or an analysis method suited to that content.

Separate a container from its contents

Consider this JSON record:

{"review_id": 1047, "rating": 4, "text": "Fast delivery, but the box was damaged."}

The keys label the values, so the record is semi-structured. The identifier and rating can become columns in a table. The text remains a freely written comment. Putting it in a spreadsheet cell does not turn its meaning into a ready-made category.

A spreadsheet can sort comments alphabetically, but that will not tell you how many customers complained about packaging. You need a field that records that idea, such as mentions_packaging, with a clear rule for assigning it.

Turn comments into countable fields

A team reads 200 reviews and marks every topic each review mentions. It records a delivery flag and a price flag for each review. There are 84 delivery mentions and 46 price mentions:

• Delivery: 84 out of 200, or 42%.
• Price: 46 out of 200, or 23%.

One review can mention both topics, so these are overlapping groups. Their percentages need not total 100%. If the team instead wants one main topic per review, it needs a rule for choosing that topic; this is a different coding task.

The tags are structured data derived from the original text. Keep the original review available to authorized analysts so that someone can check why a tag was assigned. A short coding definition, such as "mentions a delivery delay or arrival time," helps people apply the same category consistently.

Use metadata without confusing it with the content

Metadata describes data. A photo can have a capture time, camera model, and GPS location. The image content is unstructured; its capture time is a structured value that can be grouped by day. A file can also have an author, size, and creation date, and a table can have descriptions of its columns.

Structure and quality answer different questions. A carefully maintained audio archive is still unstructured. A spreadsheet with wrong quantities is still structured. Evaluate how the content is arranged separately from whether it is accurate, complete, and appropriate for the analysis.

A value's meaning decides which operations make sense. Digits alone cannot tell you whether something is a quantity, an identifier, or an ordered category.

Classify what a field means

Quantitative data counts or measures something. Qualitative, or categorical, data names a quality or group.

Delivery time is continuous even if recorded in whole minutes; the recording precision does not turn the underlying measurement into a count. A product code such as "300" is nominal because it labels a product. Adding product codes would not produce a meaningful total.

Keep category codes in context

A satisfaction scale can code "very dissatisfied" as 1 and "very satisfied" as 5. Counts, percentages, and the middle ordered response help describe it. An average of the codes assumes equal steps between neighboring categories. A difference from 1 to 2 may not represent the same change in feeling as a difference from 4 to 5.

When a field called status contains 3, ask what 3 means. It could be a count, a store label, or a rating. The data dictionary or data owner supplies that meaning.

Choose a storage type

Storage types tell software how to preserve and handle values. Common types are text or string, integer, decimal, date, timestamp, and Boolean. A Boolean has the two values TRUE and FALSE. Database names such as INTEGER, DECIMAL, VARCHAR, DATE, TIMESTAMP, and BOOLEAN vary by system.

An online shop might choose:

• order_id: text; a nominal identifier.
• order_date: date; a calendar value.
• items: integer; a discrete quantity.
• parcel_kg: decimal; a continuous quantity.
• payment_method: text; a nominal category.
• rating: an ordered category coded from 1 to 5.
• is_member: Boolean.

A rating can be stored as an integer without becoming a measured quantity. Statistical meaning and storage type are related decisions, but they are not the same label.

Prevent silent changes

Postal codes and phone numbers belong in text fields. Storing the US ZIP code "02134" as a number produces 2134 and loses part of the identifier. A longer numeric identifier can also lose precision in spreadsheet software.

Dates stored as text may sort by their first character. In that order, "10/02/2025" precedes "9/15/2025" because "1" precedes "9," even though September comes first. Use a date type when the field represents dates.

A number stored as text can be skipped by SUM when a spreadsheet adds a cell range. The total may look plausible and still omit a value. Check both the meaning of the field and the storage type before calculating, sorting, or comparing it.

A file's format determines what it can hold and how software reads it. Check those capabilities before choosing an export or opening an unfamiliar file.

Match the format to the information

CSV writes field values as text, not spreadsheet formulas. TSV uses tabs between fields. Some spreadsheet exports use semicolons where a decimal comma is common. Inspect the delimiter and encoding in an import preview.

Read quoted CSV fields

A CSV field containing a comma, a double quote, or a line break is enclosed in double quotes. A quote within that field is doubled:

1047,"Lee, Ana","She said ""great"""

This record has three fields. The comma in the name belongs to the name. The doubled quotes preserve the quotation marks around "great." CSV variants exist, so confirm how the receiving program reads the file.

JSON uses keys and values, as in {"city": "Lyon", "stations": 3}. XML uses tags, as in <city>Lyon</city>. Lists inside either format require a decision about which item becomes a row.

Preserve characters and identifiers

Character encoding maps bytes to characters. UTF-8 supports writing systems worldwide. Reading UTF-8 with the wrong encoding can turn "São Paulo" into "São Paulo." Re-import using the correct encoding; editing the visible garbling hides the cause.

Automatic spreadsheet conversion creates another risk. A code "02134" can lose its zero, long numeric IDs can display in scientific notation, and Excel stores only 15 significant digits. A 16-digit identifier can therefore lose precision. Import identifiers as text. Text that resembles a date can also change: repeated conversion of gene symbols such as SEPT2 contributed to renaming them, for example SEPTIN2, in 2020.

Check dates, rows, and conversions

The date "03/04/2025" can mean March 4 or April 3. ISO year-month-day, such as "2025-03-04," avoids that ambiguity and sorts chronologically as text when consistently formatted.

The old XLS format allows 65,536 rows per sheet. In October 2020, an import into that format contributed to 15,841 positive COVID-19 test results being omitted from England's daily figures for September 25 through October 2. Compare imported and source row counts so silent truncation is detected.

Renaming sales.xlsx to sales.csv leaves the underlying format unchanged. Convert with export or "Save as." Keep the original file: opening a CSV and saving it again can write back stripped zeros or changed dates even when you made no deliberate edits.

A table should make it easy to tell what one row describes. Start by naming that observation: an order, a customer, or a store's sales during one quarter.

Give each value a clear place

A tidy layout has one variable per column, one observation per row, and one value per cell. Use a single header row. Keep customers and orders in separate tables because they are different kinds of observations.

Merged cells, decorative blank rows, and several values in one cell make sorting and filtering unreliable. Store mass as a number under parcel_kg, not as text such as "12 kg." Record status in a field; color alone cannot reliably carry its meaning into another tool.

Keep total rows outside the records. Suppose three stores have sales values 120, 80, and 100. Their total is 300. If a fourth row also contains 300 labeled "Total," summing the whole column gives 600. The total has been counted alongside its own parts.

Choose wide or long layout

Wide layout makes side-by-side reading convenient. Long layout makes filtering, grouping, and charting by period convenient. A fifth quarter adds rows to the long table instead of changing its column structure. Pivoting turns long into wide; unpivoting turns wide into long.

Neither layout changes the sales themselves. What changes is the definition of a row and where the period is recorded. Document that row meaning so repeated store IDs in the long table are understood correctly.

Explain the fields

A data dictionary describes each field's meaning, type, unit, allowed values or range, and missing-value code. For example:

trip_km: Trip distance in kilometers; decimal; zero or greater; blank when the GPS signal failed.

This entry makes a blank different from a measured zero. It also prevents a reader from treating kilometers as meters. A README supplies the dataset overview: its source, coverage, time period, and known problems.

Descriptive metadata identifies the dataset, author, and subject. Structural metadata describes fields, tables, and relationships. Administrative metadata records ownership, access rights, licenses, creation dates, and file types. Together they help people locate and judge data as well as use it.

Leave a trace of your work

Use meaningful names in one style, such as parcel_kg and order_date. A file name such as bike_trips_2025-03-04_v02.csv records content, date, and version without spaces or special characters. Year-month-day dates make related file names sort in time order.

Keep raw data unchanged and read-only, with working copies and outputs in separate folders. Record changes in a short log or shared version history. Write definitions while the work is fresh: an unexplained status value of 3 can leave the next analyst unable to interpret an otherwise clean table.

Choose a data store for the amount of data, the relationships it contains, and the people who need to read or change it. A convenient display is only one part of that choice.

Understand linked tables

A database is an organized electronic collection managed by a database management system, or DBMS. The DBMS handles storage, permissions, and consistency. A relational database keeps related facts in tables linked through keys. Its schema defines tables, fields, types, and relationships.

A primary key uniquely identifies a row and is never empty. It can be one field or a combination. A foreign key links a table to another table's primary key. Several records can point to the same related record.

Consider customers, with customer_id, name, and city, and orders, with order_id, customer_id, order_date, and items. Customer C102 has three orders. The identifier C102 appears three times in orders, but only once as a primary key in customers. Each order keeps its own unique order_id.

If the customer's current city changes, updating the customer row makes the new city available through the relationship. Repeating the city on every order would invite conflicting edits. Organizing related facts in linked tables to reduce repetition and contradictions is normalization. Use stable IDs: names can repeat or change.

Compare working needs

An Excel worksheet holds at most 1,048,576 rows and 16,384 columns. Google Sheets allows up to 20 million cells per spreadsheet, increased from 10 million in September 2026. Capacity is a ceiling, not a promise of smooth performance for every file.

Make the size check concrete

A file of 2,500,000 trips needs about 2.4 times the rows of one Excel worksheet. Store it in a database and extract a useful subset. A table of 700,000 rows and 30 columns uses 21,000,000 cells, exceeding the Google Sheets limit. A list of 300 event registrations can fit comfortably in a spreadsheet.

Even small data may need a database when many people update related records and consistent keys matter. Spreadsheets can reject bad entries with validation rules, but those rules need setup and may be bypassed, including during pasting. A table in a sheet does not acquire database constraints simply from its layout.

Recognize other stores

A document-oriented NoSQL database can store JSON records with varying fields. A data warehouse combines prepared data from several sources for analysis. A data lake keeps large volumes of raw data in original formats, including media. A common workflow keeps the main data in a database and sends a selected subset to a spreadsheet or business intelligence tool.

Credibility means a source is trustworthy and its collection and documentation support reliance on it. A polished file is easy to read; the evidence behind it determines how much confidence it deserves.

Apply ROCCC

ROCCC stands for Reliable, Original, Comprehensive, Current, and Cited. Treat each word as a question to investigate.

Original does not mean you must collect everything yourself. A reused table can be credible if you can follow it to a documented original. Current does not mean newest at any cost: historical records are the right records for a historical question.

Trace a claim before accepting it

A forum post links a spreadsheet called "Average commute time by city." It has no author, date, or definition of "commute." You cannot tell whether its figures describe one-way journeys, round trips, workers, or everyone in the city.

First, look for the originating statistics office or transport authority. Find its table and methodology. Check the year and the definition of the measure, then confirm that the copied figures match. If you cannot recover that trail, the forum file leaves important trust questions unanswered.

A source can be reliable and still not answer your question. A travel survey of employed adults may suit a question about work journeys but omit trips made by retired people. Comprehensive always refers to the question and the people it asks about.

Follow provenance and permissions

Provenance is the record of where data originated and what happened to it. A useful trail connects the original release, the downloaded version, any reshaping or filtering, and the extract used in a result. Metadata and a data dictionary explain fields and definitions along that trail.

Check the license or agreement before reuse. Permission to download a file may differ from permission to publish its rows or use it commercially. A technically sound source can still be unusable for your planned purpose under its terms.

Cross-check meaning as well as totals

Compare regional totals with the publisher's national total when the regions cover that whole population and period. A mismatch is a reason to investigate missing areas, different dates, revisions, or different definitions. Compare an independent source too; several websites repeating one figure provide one evidence trail, not independent confirmation.

Read revision notes. A later correction can make a saved extract stale even if its download was recent. Record the release you used and the date you accessed it. When a dashboard says "active user," find out whether that means a login, a purchase, or another action before comparing it with a second dashboard.

Bias is a systematic distortion in how data is gathered, recorded, selected, or interpreted. It pushes a result in a direction. More records collected in the same biased way preserve that distortion.

Identify what went wrong

Observer bias concerns the recording process; interpretation bias concerns the meaning someone gives the information. Confirmation bias is especially visible in which evidence someone seeks or keeps. More than one can occur in a project, but name the mechanism the evidence actually supports.

Work out the effect of missing customers

A survey gets 300 app responses, of which 240 are satisfied. The observed satisfaction rate is 80%. App users make up 40% of all customers. A separate check finds that 55% of the remaining 60% are satisfied.

To combine these group rates, account for each group's share:

• App contribution: 40% of 80%, or 32 percentage points.
• Other customers' contribution: 60% of 55%, or 33 percentage points.
• Overall satisfaction: 32 plus 33, or 65%.

Reporting 80% for all customers would extend the app result to a group it did not represent. An even larger app-only survey would still miss that group.

Look for gaps and skewed choices

Compare the dataset's makeup with known population figures, such as age-group shares or customer channels. Ask who is absent, who dropped out, and why. Review how categories were defined and whether a response option leaves people out. Examine who recorded judgments and what expectations could have affected them.

An accurate record of past decisions deserves a separate fairness check. If a hiring process favored one group, accurately storing its decisions preserves that pattern. Correct transcription does not repair the unequal history.

Reduce the mechanism you found

Clear recording definitions, standard procedures, and independent observers help reduce observer bias. Agreed criteria for interpreting results reduce room for expectations to drive conclusions. Writing the question and success criteria before analysis, and actively seeking evidence against your idea, counters confirmation bias.

Document biases you cannot remove and explain how they restrict conclusions. If canceled customers are absent, describe the result as a result about current subscribers. If the decision requires understanding cancellation, seek information about those who left.

Data quality means fitness for an intended use. Assess the fields and records the decision actually needs, then decide what the problems would do to the result.

Inspect six dimensions

Validity and accuracy differ. An age of 42 can fit the allowed range while being wrong for a person whose real age is 24. Accuracy requires a comparison with reality, not just a format check.

Profile a customer table

A table has 5,000 rows, 600 blank email addresses, 45 customer IDs that appear more than once, a maximum age of 212, and an update date 18 months ago.

1. Divide 600 by 5,000: 12% of email entries are missing.
2. Repeated IDs raise a uniqueness problem if one row should describe one customer.
3. Age 212 fails a plausible age-range rule and needs investigation.
4. The update date raises a timeliness concern for current customer contact.

This profile locates problems; it does not decide every use. A regional spending analysis for last year may proceed after purchase amounts, dates, and regions are verified, duplicate records are resolved, and the full requested year is covered. An email campaign needs current email addresses and is affected directly by the missing contact field.

Decide what evidence is sufficient

Data is insufficient when the question needs records, fields, coverage, or a period that is absent. A single source deserves extra checking when you cannot cross-check it. A changing source needs a fixed snapshot or reporting cutoff, such as "all orders up to 2025-06-30," so someone can reproduce the result.

When a gap matters, discuss the choice with stakeholders. You can seek new data, wait, narrow the question, report a limited result, or use a clearly explained proxy. Proxy data stands in for something unavailable. Early sales of a similar product might inform a new product estimate, but the similarity is an assumption to state and examine.

Check the effect of your response

Deleting every row with a blank can introduce bias. If older customers are more likely to omit an email address, deleting those rows underrepresents them even in an analysis that does not need email.

A file with no blanks can still describe the wrong population or use the wrong definition. Record which problems affect the planned decision, what needs resolution, and which limits remain.

Personal data relates to someone who is identified or can be identified. The label personally identifiable information, or PII, is also common. In everyday analysis, look at what the information reveals and how it could be linked to a person.

Look for direct and indirect identification

A direct identifier can point straight to a person or account: an email address, phone number, national ID, account number, or clear face photo. Names also help identify people, although a common name may need context.

An indirect identifier, or quasi-identifier, becomes identifying in combination with other details. Birth date, postal code, gender, employer, and job title may seem broad separately but leave a very small group when combined.

Latanya Sweeney estimated from 1990 US census data that about 87% of the population was probably unique on five-digit ZIP code, gender, and full birth date. A later study using 2000 census data estimated about 63%. These are findings tied to those populations and dates, not a universal identification rate. Sweeney also linked a state health dataset with a voter list to identify a governor's records.

Inspect every field, including notes

Consider these customer fields:

A note such as "Ask for Maria at the Elm Street branch, she knows my case" contains a name and a location despite the field's generic label. Read free-text fields before deciding an extract is safe to share.

Device IDs, cookie IDs, IP addresses, precise movement histories, and GPS metadata can also support identification. A route beginning at the same house each morning can reveal where a person lives even when the route record has no name.

Recognize sensitive information

Sensitive data is personal information whose misuse can cause serious harm. Examples include health, genetic and biometric data; racial or ethnic origin; religious and political beliefs; sexual orientation; union membership; financial details; precise location; and information about children. A task should have a clear need for such fields before they enter an analysis copy.

Identification and sensitivity are different concerns. An email can identify a person without describing their health. A diagnosis may be sensitive even when its table has replaced names with codes, because the records may remain linkable.

Respect the use as well as the source

Privacy concerns people's control over information about them and whether collection, use, and sharing are appropriate and expected. Public profiles and directories still describe people. Combining public details can reveal more than the original publishers showed together.

Examples of privacy laws include the EU GDPR, UK GDPR, and California CCPA. At work, follow the organization's privacy policy and consult its privacy or legal team about uncertain uses. Removing a name is one change to a dataset; judging what the remaining information can reveal requires inspecting the whole extract and possible linked sources.

De-identification removes or changes details to make records harder to link to people. The right method preserves enough information for the question while reducing what a recipient can learn about individuals.

Distinguish codes from anonymity

Pseudonymization replaces identifiers with codes. A separate lookup key can reconnect them, so the data remains personal. Anonymization requires that people can no longer reasonably be identified, taking account of means likely to be used, and that the transformation cannot be reversed.

Plain hashing of email addresses is pseudonymization: someone can hash a list of candidate addresses and compare the results. Random IDs or a protected secret-key method offer safer pseudonyms, while the lookup information still needs separate protection.

Choose a transformation

Each choice loses some detail. Age bands may answer a segment question without exposing exact ages, but cannot show variation within a band.

Work through k-anonymity

k-anonymity checks groups formed by the quasi-identifiers you selected. Every combination must appear at least k times. The smallest group's size is the table's k.

Eight patient records contain age/postal-code pairs: 34/10115, 36/10117, 38/10119, 31/10115, 52/10243, 57/10245, 55/10247, and 59/10249. Each complete pair is unique, so k is 1.

Generalize age into ten-year bands and postal code into its first three digits:

• Four records now share age band 30–39 and postal prefix 101.
• Four records share age band 50–59 and postal prefix 102.

The smallest group has four records, so the result is 4-anonymous on those quasi-identifiers. Group size uses all selected quasi-identifiers together, not separate counts for each column.

Check what the groups still reveal

If all four people in a group share one diagnosis, knowing that a person belongs to that group reveals the diagnosis without finding their row. Group size alone does not prevent disclosure of a sensitive value. Other datasets can also create linkage opportunities. In 2008, researchers linked movie ratings and approximate dates to public reviews despite names having been replaced with numbers.

Aggregates need review too. A count of one can expose someone. Two overlapping pay totals can reveal a difference: 720,000 for 12 employees minus 655,000 for the 11 who are not new hires reveals the one new hire's pay of 65,000.

List the fields, remove those the question does not need, suppress direct identifiers, protect any lookup key, generalize quasi-identifiers, and inspect small groups and free text. Check overlapping releases as well as the file in front of you, and document the transformations and remaining linkage risks.

Security protects data from unauthorized access, alteration, and loss. Privacy asks whether a use is appropriate. A locked file can still be used intrusively, and a respectful analysis plan still needs a secure place to work.

Limit who can do what

Access control specifies who can read or change data. Least privilege means giving each person only the access needed for their task and only for the necessary time. Someone reviewing aggregate results may need no access to the individual records used to create them.

Share with named recipients, choose read or edit rights deliberately, and remove access at the project's end. An "anyone with the link" setting can allow the link to travel beyond the intended audience. A project folder's convenience does not justify giving every project member every source file.

Protect stored and moving data

Encryption makes data unreadable without the correct key. Use protection at rest, for stored files, devices, and backups, and in transit, while data moves between systems, for example over HTTPS. Check both parts: protected storage does not protect a copy sent through an unsuitable transfer channel.

Tokenization substitutes a token for a sensitive value while keeping the real value in a separately secured system. It reduces where the original value needs to appear. The system holding the original still needs strong controls.

Reduce the amount and the lifetime

Data minimization means collecting, copying, and keeping only the personal information the task needs. Extract five necessary columns instead of a complete customer table containing names, addresses, and unrelated notes. A smaller extract exposes less if it is mishandled.

Retention rules set how long to keep data and when to delete it. Apply them to working copies and shared extracts, not just the main database. Revoke permissions when access ends, and follow approved deletion procedures for copies.

Check the file that will actually leave

Hidden columns and sheets can be unhidden. Sheet protection can prevent edits while leaving values readable. Version history may retain removed values, and file properties may reveal an author's identity. Build a clean extract instead of relying on display tricks to hide information.

For customer-cancellation results sent to an outside agency, a suitable plan is to share rates by region and age band in groups of at least 10 customers, using the approved transfer service. Give access to two named agency staff and expire it after 30 days. The group threshold in this plan is a chosen safeguard, not a guarantee against every disclosure.

Use approved storage and transfer tools. Personal email, portable drives, or unapproved online tools can create uncontrolled copies. Do not paste personal or confidential records into unapproved AI assistants or other web tools. If a file goes to the wrong person or an unencrypted device is lost, report it immediately through the organization's incident procedure so the response team can act.

Data ethics provides standards for judging whether collecting, using, and sharing data is fair, honest, and respectful. Legal rules set a minimum. Ethical judgment also asks what a decision does to the people behind the records.

Understand people's stake in their data

Ownership, as an ethical principle, recognizes the primary claim of people to control how data they provide is used and shared. Keeping a copy on a company server does not remove their interests.

Consent should be informed, specific to a purpose, freely given, and possible to withdraw. People need to understand how and why their data will be used before agreeing. A vague checkbox buried in long terms does not make an unexpected use understandable. Check a new purpose again; an earlier agreement does not settle all later uses.

Make processing and money visible

Transaction transparency means people can understand how their data is processed, including algorithms that act on it. If an automated score refuses someone a discount, a plain-language explanation should identify what the score is based on.

Currency concerns awareness of financial transactions resulting from personal data and their scale. A person may understand that an app records location without realizing the company sells that history to advertisers. Transparency about collection and transparency about money answer different questions.

Balance sharing, privacy, and fairness

Open data is accessible and reusable under a clear license. CC BY 4.0 allows reuse with attribution. CC0 is a public-domain dedication intended to remove copyright restrictions as far as possible. Open reuse still requires attention to personal information and possible harm.

Accountability needs records: identify sources, methods, decisions, and the person responsible. Crediting a dataset lets a reviewer follow the evidence and gives its creator appropriate recognition.

Apply the principles to a fitness app

A free fitness app sells precise running routes to advertisers. Its long terms mention only that "partners" may receive location data. The routes can reveal homes and daily routines.

• Consent is weak because the specific use was not made clear before agreement.
• Transaction transparency is weak because people cannot readily see what processing and sharing occur.
• Currency is weak because the sale and its scale are hidden from users.
• Privacy is threatened because detailed routes disclose personal routines.

The company should examine whether the sharing serves a purpose people understand, whether a less revealing alternative meets that purpose, and whether users can make a meaningful choice. Calling a use lawful does not answer those ethical questions.

Ethics is part of framing the question, preparing evidence, analyzing it, reporting results, and acting on them. The analyst often sees a harmful pattern before anyone reviewing a final presentation does.

Work through a decision

1. State the purpose and the data. What decision is being supported, and what information would be used?
2. Identify affected people and effects. Include people missing from the file as well as those represented. Name possible benefits and harms.
3. Check expectations and obligations. Compare the use with what people were told, their agreements, organizational policy, and applicable law.
4. Compare alternatives. Could less data, broader groups, or opt-in participation answer the real need?
5. Decide and document. Record the reasons, the responsible person, and any limits. Escalate high risks or unresolved questions.

Examine a sensitive inference

A retailer wants to infer pregnancy from shopping histories and send baby-product coupons. The commercial purpose does not require customers to reveal a health-related inference. A coupon delivered to a shared home could disclose a pregnancy to family members, including when the inference is wrong.

Check what customers expected when purchases were recorded. Consider an opt-in baby club as a less intrusive way to send relevant offers. Document why that alternative better respects people's choices and avoids making an unexpected sensitive inference.

A workplace case poses a related decision: HR wants a resignation-risk score based on badge swipes and email metadata. Ask whether employees expected this use, how false flags could affect careers, and whether part-time work or medical leave could be penalized. An anonymous engagement survey may address the underlying concern with less intrusion.

Test fairness beyond one column

A proxy variable stands in for another characteristic. Removing ethnicity from a scoring file may leave postal code or first language strongly associated with it. The remaining fields can preserve unequal treatment. Check outcomes for affected groups and investigate the process behind differences.

This fairness question differs from using proxy data to estimate an unavailable measure. Both involve a stand-in, but one asks about indirect group effects and the other about the adequacy of substitute evidence.

Check access and reporting choices

Before scraping public profiles, examine terms, licenses, personal information, and people's expectations. Access to a web page gives you the ability to read it; permission for a marketing database is a separate question.

Report evidence honestly, including results that weaken your preferred explanation. Use charts that represent differences fairly, state uncertainty and coverage limits, and credit sources. Selectively showing only favorable results can turn a careful analysis into a misleading recommendation.

Anonymous data still supports decisions about groups. A result can stigmatize a neighborhood or deny it services even if nobody's row is identifiable. Assess what the proposed action would do to that group, assign someone to review its effects, and raise concerns early with the manager or data owner.