import bigframes.pandas
import pandas as pd
from bigframes.ml.llm import GeminiTextGenerator
Define the model#
model = GeminiTextGenerator(model_name="gemini-2.0-flash-001")
/tmp/ipykernel_176683/987800245.py:1: ApiDeprecationWarning: gemini-1.5-X are going to be deprecated. Use gemini-2.0-X (https://cloud.google.com/python/docs/reference/bigframes/latest/bigframes.ml.llm.GeminiTextGenerator) instead.
model = GeminiTextGenerator(model_name="gemini-2.0-flash-001")
/usr/local/google/home/shuowei/src/python-bigquery-dataframes/bigframes/ml/llm.py:486: DefaultLocationWarning: No explicit location is set, so using location US for the session.
self.session = session or global_session.get_global_session()
Query job 6fa5121a-6da4-4c75-92ec-936799da4513 is DONE. 0 Bytes processed. Open Job
Query job 74460ae9-3e89-49e7-93ad-bafbb6197a86 is DONE. 0 Bytes processed. Open Job
Create Sample Data#
Read as a BigQuery DataFrames.
df = pd.DataFrame(
{
"prompt": ["What is BigQuery?", "What is BQML?", "What is BigQuery DataFrame?"],
})
bf_df = bigframes.pandas.read_pandas(df)
Make Predictions#
pred = model.predict(bf_df).to_pandas()
pred
Query job 562ca203-3b53-4409-9a23-0a80d3840fcc is DONE. 0 Bytes processed. Open Job
/usr/local/google/home/shuowei/src/python-bigquery-dataframes/bigframes/core/array_value.py:114: PreviewWarning: JSON column interpretation as a custom PyArrow extention in
`db_dtypes` is a preview feature and subject to change.
warnings.warn(msg, bfe.PreviewWarning)
Query job 5a6ceff2-53b5-4a4a-83ff-31bffab1b8b8 is DONE. 14.0 kB processed. Open Job
| ml_generate_text_llm_result | ml_generate_text_rai_result | ml_generate_text_status | prompt | |
|---|---|---|---|---|
| 0 | BigQuery is a serverless, highly scalable, and... | <NA> | What is BigQuery? | |
| 1 | BQML stands for **BigQuery Machine Learning**.... | <NA> | What is BQML? | |
| 2 | BigQuery DataFrames is a Python client library... | <NA> | What is BigQuery DataFrame? |
Fetch Predictions#
pred.iloc[0, 0]
"## BigQuery: A serverless data warehouse for large-scale data analysis\n\nBigQuery is a serverless, highly-scalable data warehouse designed for analyzing large datasets. It's a cloud-based service offered by Google Cloud Platform (GCP), allowing users to store, manage, and analyze massive amounts of data without managing infrastructure. \n\nHere are some key features of BigQuery:\n\n**Serverless:** You don't need to worry about provisioning, managing, or scaling servers. BigQuery handles all of this automatically, letting you focus on analyzing your data.\n\n**Highly-scalable:** BigQuery can handle datasets of any size, from gigabytes to petabytes. It can also scale up and down automatically to meet your processing needs.\n\n**Cost-effective:** You only pay for the resources you use, and there are no upfront costs. Additionally, BigQuery offers several pricing models to fit your needs, including on-demand, flat-rate, and flexible slots.\n\n**Easy to use:** BigQuery uses SQL, a standard query language, making it easy to analyze your data. No need to learn a new programming language.\n\n**Integrated with GCP:** BigQuery integrates seamlessly with other GCP services, such as Google Cloud Storage, Dataflow, and Kubernetes. This allows you to build powerful data pipelines and workflows.\n\n**Secure:** BigQuery uses industry-standard security practices to protect your data.\n\nHere are some use cases for BigQuery:\n\n* **Data warehousing and analytics:** Store and analyze large datasets for business intelligence and reporting.\n* **Machine learning:** Train and deploy machine learning models on your data.\n* **Data integration:** Combine data from multiple sources for analysis.\n* **Real-time analytics:** Analyze data in real-time for insights and decision-making.\n\n**Here are some additional resources that you may find helpful:**\n\n* **BigQuery website:** https://cloud.google.com/bigquery\n* **BigQuery documentation:** https://cloud.google.com/bigquery/docs\n* **BigQuery tutorial:** https://cloud.google.com/bigquery/docs/tutorials\n* **BigQuery pricing:** https://cloud.google.com/bigquery/pricing\n\nI hope this gives you a good overview of BigQuery. Please let me know if you have any other questions."