Quarto: Python and Observable JS

Author

Qian

Published

November 16, 2023

Welcome to the intersection of innovation and interactivity in scientific blogging! In this blog, we explore how Quarto leverages the strengths of Python and Observable JavaScript to transform the way we create and share data-driven narratives. Dive into the world of dynamic content creation where Pythonโ€™s robust analysis capabilities meet the interactive flair of Observable JS.

What is Quarto?

Quarto is an open-source scientific and technical publishing system that supports creating dynamic and reproducible content. It allows authors to write in Jupyter notebooks or use plain text markdown in their preferred editor. Quarto facilitates the creation of various forms of content, including articles, presentations, dashboards, websites, blogs, and books, which can be published in multiple formats such as HTML, PDF, MS Word, ePub, and more.

Key features of Quarto include:

  • Compatibility with multiple programming languages, including Python, R, Julia, and Observable for dynamic content creation.
  • The ability to publish high-quality, production-level documents.
  • Support for sharing knowledge and insights across an organization by integrating with systems like Posit Connect, Confluence, and other publishing platforms.
  • Advanced writing capabilities using Pandoc markdown, which includes support for equations, citations, cross-references, figure panels, callouts, and advanced layout options.

Quartoโ€™s versatility makes it an ideal tool for academics, data scientists, and technical writers who need to produce detailed and interactive documentation.

Comparing Python and Observable Javascript(OJS) in Quarto

In the introduction section, we highlighted the versatility of Quarto in accomodating various progrqmming languages to craft dunamic content. This versatility brings us to an intriguing comparison between two prominent dynamic content: Python and Observable JS. Each language brings its unique strengths and applications in the realm of Quarto. In the forthcoming sections, I will delve into the practical application of Python and OJS in developing a Quarto blog. This discussions aims not only to showcase their individual capabilities but also to elucidate the distinct characteristics and advantages they offer.

Python in Quarto

Python is a powerful programming language widely used for data manipulation and analysis with its rich array of libraries. Precisely, Python is suited for complex, concrete and static visualizations where the focus is on data representation without great need for interactivity. The belowing is a comparison of a simple data processing by using these two languages.

### data processing in Python

import pandas as pd
import plotly.express as px
import os
import plotly.io as pio
import plotly.io as pio

pio.renderers.default = "notebook"
pio.templates.default = "plotly_dark"

# Sample data
data = {
    'Month': ['January', 'February', 'March', 'April', 'May', 'June'],
    'Sales': [200, 150, 300, 250, 400, 320]
}
df = pd.DataFrame(data)

px.bar(
    df, x='Month', y='Sales', color='Sales', 
    labels={'Sales':'Sales Figures'}, 
    title='Monthly Sales Data',
    color_continuous_scale="YlGnBu_r"
)
// data processing in OJS
{
  const data = [{name: "Product A", value: 200}, {name: "Product B", value: 150}, {name: "Product C", value: 300}];
  const width = 500, height = 300, margin = {top: 20, right: 30, bottom: 40, left: 90};

  const svg = d3.create("svg")
      .attr("viewBox", [0, 0, width, height]);

  const x = d3.scaleBand()
      .domain(d3.range(data.length))
      .range([margin.left, width - margin.right])
      .padding(0.1);

  const y = d3.scaleLinear()
      .domain([0, d3.max(data, d => d.value)]).nice()
      .range([height - margin.bottom, margin.top]);

  svg.append("g")
    .attr("fill", 'steelblue')
    .selectAll("rect")
    .data(data)
    .join("rect")
      .attr("x", (d, i) => x(i))
      .attr("y", d => y(d.value))
      .attr("height", d => y(0) - y(d.value))
      .attr("width", x.bandwidth());

  svg.append("g")
      .attr("transform", `translate(0,${height - margin.bottom})`)
      .call(d3.axisBottom(x).tickFormat(i => data[i].name));

  svg.append("g")
      .attr("transform", `translate(${margin.left},0)`)
      .call(d3.axisLeft(y).ticks(null, data.format));

  return svg.node();
}

After comparing data processing and visualization capabilities between Python and Observable JavaScript (OJS) in Quarto, it is evident that Python offers a more straightforward and clearer approach. The Python code, utilizing libraries like Pandas and Plotly, provides a simplified yet powerful method for data manipulation and creating concrete visualizations. The rich libraries cover a wide range of functionalities needed in programming, especially for scientific computing, machine learning, and data analytics, areas where JavaScript lacks similar inbuilt libaries. This makes Python a more comprehensive choice for various data analysis applications. [OpenXcell - JavaScript Vs Python - Detailed Comparison]

Observable Javascript (OJS) in Quarto

OJS in quarto shines in creating highly interactive, real-time web visualization that can intergrate seamlessly with modern web technologies. Precisely, OJS is ideal for users to creat dynamic and responsive web-based visualization. Moreover, we can easily intergrate with JS for custom intergrativity, making our creations more suitable for web-based iteractive visualizations.

Iโ€™ll next demonstrate some dynamic graphs and animation with help of OJS.

Exemple 1 : Emoji custom visualization (Python+OJS)

# This part of code is written in Python as a simple data processing.
data = [
  { "animal": "pigs", "country": "Great Britain", "count": 1354979 },
  { "animal": "cattle", "country": "Great Britain", "count": 3962921 },
  { "animal": "sheep", "country": "Great Britain", "count": 10931215 },
  { "animal": "pigs", "country": "United States", "count": 6281935 },
  { "animal": "cattle", "country": "United States", "count": 9917873 },
  { "animal": "sheep", "country": "United States", "count": 7084151 }
]

df_animal = pd.DataFrame(data)

# save
df_animal.to_csv("animals.csv")


df_animal
animal country count
0 pigs Great Britain 1354979
1 cattle Great Britain 3962921
2 sheep Great Britain 10931215
3 pigs United States 6281935
4 cattle United States 9917873
5 sheep United States 7084151

In the following code, we map animal types to emojis and use Plotly to create a text-based plot where the frequency of each animal type in different countries is represented by repeating emojis. By using OJS, this plot is customized in terms of size, layout and aesthetics.

data = FileAttachment("animals.csv").csv({ typed: true })
emoji =  ({ cattle: "๐Ÿ„", sheep: "๐Ÿ‘", pigs: "๐Ÿ–" })

Plot.plot({
  width: 610,
  height: 380,
  marginLeft: 60,
  marginRight: 100,
  marginTop: 30,
  y: {label: null},
  fy: {paddingInner: 0.27, label: null},
  style: {
    background: 'transparent' // Set the background color to transparent
  },
  marks: [
    Plot.text(data, {
      frameAnchor: "left",
      fontSize: 40,
      text: (d) => `${emoji[d.animal]} `.repeat(Math.round(d.count / 1e6)),
      dx: 20,
      y: "animal",
      fy: "country"
    }),
  ],
  caption: "Live stock (millions)"
})

By the way, you can choose other emoji animals as you want :) Type windows + "." in order to display emoji keyboard

Exemple 2 :Images Plot(Python+OJS)

# You can download this dataset from the reference link and handle its data as your want. In this part, I just read the original dataset with the library pandas in Python.
pd.read_csv("./us-president-favorability@1.csv").head(5)
Name Very Favorable % Somewhat Favorable % Somewhat Unfavorable % Very Unfavorable % Donโ€™t know % Have not heard of them % First Inauguration Date Portrait URL
0 George Washington 44 26 6 4 18 3 1789-04-30 https://upload.wikimedia.org/wikipedia/commons...
1 John Adams 16 30 7 4 37 5 1797-03-04 https://upload.wikimedia.org/wikipedia/commons...
2 Thomas Jefferson 28 34 10 5 23 1 1801-03-04 https://upload.wikimedia.org/wikipedia/commons...
3 James Madison 12 27 5 4 43 9 1809-03-04 https://upload.wikimedia.org/wikipedia/commons...
4 James Monroe 8 21 8 4 49 10 1817-03-04 https://upload.wikimedia.org/wikipedia/commons...

The following code is an another custom plot example written in OJS. We plot images (portraits of US presidents) along the x-axis, which represents their first inauguration dates.
Source of code and dataset: ObservableHQ Plot Image Dodge

presidents = FileAttachment("us-president-favorability@1.csv").csv({typed: true})
Plot.plot({
  inset: 20,
  height: 280,
  style: {
    background: 'transparent' // Set the background color to transparent
  },
  marks: [
    Plot.image(
      presidents,
      Plot.dodgeY({
        x: "First Inauguration Date",
        r: 20, // clip to a circle
        preserveAspectRatio: "xMidYMin slice", // try not to clip heads
        src: "Portrait URL",
        title: "Name"
      })
    )
  ]
})

Exemple 3 :Animations plot (D3.js in OJS)

The following visualization includes dynamic, smooth zooming effects on random circles, showcasing the capabilities of D3.js in Observable JavaScript for creating interactive and animated data visualizations.
Reference link : D3.js smooth zooming

theta = Math.PI * (3 - Math.sqrt(5))
step = radius * 2
radius = 6
height = 500 // Observable provides a responsive *width*

data_zoom = Array.from({length: 2000}, (_, i) => {
  const r = step * Math.sqrt(i += 0.5), a = theta * i;
  return [
    width / 2 + r * Math.cos(a),
    height / 2 + r * Math.sin(a)
  ];
})

chart = {
  let currentTransform = [width / 2, height / 2, height];

  const svg = d3.create("svg")
      .attr("viewBox", [0, 0, width, height])

  const g = svg.append("g");

  g.selectAll("circle")
    .data(data_zoom)
    .join("circle")
      .attr("cx", ([x]) => x)
      .attr("cy", ([, y]) => y)
      .attr("r", radius)
      .attr("fill", (d, i) => d3.interpolateRainbow(i / 360))

  function transition() {
    const d = data_zoom[Math.floor(Math.random() * data_zoom.length)];
    const i = d3.interpolateZoom(currentTransform, [...d, radius * 2 + 1]);

    g.transition()
        .delay(250)
        .duration(i.duration)
        .attrTween("transform", () => t => transform(currentTransform = i(t)))
        .on("end", transition);
  }

  function transform([x, y, r]) {
    return `
      translate(${width / 2}, ${height / 2})
      scale(${height / r})
      translate(${-x}, ${-y})
    `;
  }

  return svg.call(transition).node();
}

As demonstrated in the three exemples above, we can find that OJS excels in creating interactive, dynamic and visually appealing representations of data, suitable for web-based applications.

Combination of Python and OJS in Quarto

After discussing propre strenghs of these two languages, it should be noted that a combination of both of them allows us to create more dynamic and interactive data-driven applications.

Python for Backend, OJS for Frontend

  • Data Processing wih Python: We can first use Python for its powerful data analysis capabilities(Prepare, clear, handle and analyze data).
  • Visualization with OJS: After processing our data with Python, we can use OJS for creating interative visualizations, particularly with D3.js, an excellent library for buliding dynamic and responsive web-based visualizations.

To wrap up

As we wrap up our exploration of Python and Observable JavaScript (OJS) in Quarto, weโ€™ve seen firsthand how each language uniquely enhances the art of scientific blogging. Python, with its extensive libraries, anchors our narrative in robust data analysis, allowing us to delve deep into datasets and present our findings with precision and clarity.

In contrast, Observable JS introduces an element of interactivity and visual elegance, making the complex data we work with not only accessible but also engaging. It turns our static datasets into dynamic, immersive experiences, inviting readers to engage with the content in a meaningful way.

The integration of these two powerful tools in Quarto marks a new chapter in scientific blogging. Itโ€™s not just about presenting data; itโ€™s about weaving it into living, interactive stories that foster a deeper understanding and ignite curiosity.

Thanks for joining this journey into the possibilities of Quarto!

Useful ressources about Quarto

  • https://quarto.org/
  • https://observablehq.com/plot/
  • https://www.infoworld.com/article/3674789/a-beginners-guide-to-using-observable-javascript-r-and-python-with-quarto.html
  • https://observablehq.com/@d3/smooth-zooming?collection=@d3/d3-zoom
  • https://observablehq.com/@observablehq/plot-image-dodge?intent=fork
  • https://www.openxcell.com/javascript-vs-python-detailed-comparison/
Code 
{
    // The Easter Egg of our journey: the final example showcasing a custom visualization crafted in Observable JavaScript (OJS);)! 
    function createInteractiveLogo() {
        // SVG setup
        const width = window.innerWidth * 0.9;
        const height = width*0.4;

        const svg = d3.create("svg")
            .attr("viewBox", `0 0 ${width} ${height}`) // Use viewBox for responsiveness
            .attr("preserveAspectRatio", "xMidYMid meet") // Preserve aspect ratio
            .style("width", "100%") // Full width of the container
            .style("height", "auto"); // Height adjusts automatically

        // Add text for 'Qian!'
        const fontSize = Math.min(width / 10, 64);
        const text = svg.append("text")
            .text("Qian!")
            .attr("x", "50%") // Center horizontally
            .attr("y", "50%") // Center vertically
            .attr("text-anchor", "middle")
            .style("font-family", "Arial, sans-serif")
            .style("fill", "#FFC0CB") // Pink color
            .style("font-size", "64px")
            .style("cursor", "pointer");

        // Mouseover event to add interactivity
        text.on("mouseover", function() {
            d3.select(this)
                .transition()
                .duration(500)
                .style("font-size", "90px")
                .style("fill", "#FF69B4")
                .transition()
                .duration(300)
                .ease(d3.easeLinear)
                .attr("transform", `rotate(-10, ${width / 2}, ${height / 2})`)
                .transition()
                .duration(300)
                .ease(d3.easeLinear)
                .attr("transform", `rotate(10, ${width / 2}, ${height / 2})`)
                .transition()
                .duration(300)
                .ease(d3.easeLinear)
                .attr("transform", `rotate(0, ${width / 2}, ${height / 2})`)
                .style("fill", "#FFC0CB"); // Back to original color
        });

        // Mouseout event to reset the logo
        text.on("mouseout", function() {
            d3.select(this)
                .transition()
                .duration(500)
                .style("font-size", "64px")
                .style("fill", "#FFC0CB") // Original pink color
                .attr("transform", ""); // Reset transformation
        });

        return svg.node();
    }
    
    return createInteractiveLogo();
}