Intro to Comp Media: Media

Sound Sketch

Final sketch: https://editor.p5js.org/willxuan99/sketches/YZ8-jR5B2

Keywords: upbeat, casual, repetition

Insipration: We are aimming to create a light mood beat that includes daily life sound such as finger snap, triangle, phone notification sound, dog and cat. The tempo of our sound sketch is very rhythmic and fast paced. Some of the music reference are Truth or Dare by Tyla, Cherry Wine by grentperze, and ETA by New Jeans. 

And also we try to make something with an interactive sound with the mouse.(But we couldn’t reproduce the exact same sound, so we just archived it)

Interactive sound sketch: https://editor.p5js.org/jp7469/sketches/Xkt7T8eki

Inspiration from “Daisy Bell”. It was composed by Harry Dacre in 1892. In 1961, the IBM 7094 became the first computer to sing, singing the song Daisy Bell. Vocals were programmed by John Kelly and Carol Lockbaum and the accompaniment was programmed by Max Mathews. We felt the concept of this song has similarities with assignment.

Original Sound: First computer to sing – Daisy Bell


Pixel Manipulation

Final P5 Link (with image)

Initial P5 Link (Emoji pixel manipulation in real-time)

<Pixel-emoji with a picture>

  1. Overview

This program is a p5.js sketch that transforms an input image into pixel art using emojis. By analyzing color and brightness, it selects appropriate emojis and adds animation effects to create a dynamic result.

2. Core Concepts

2.1 Dynamic Resolution Change

let maxCellSize = 1000;   // Initial cellSize value
let minCellSize = 4;      // Final cellSize value
let currentCellSize = 2;

The program operates by starting with a large cell size that gradually shrinks, creating an effect like a camera zooming in, revealing more detail as it progresses.

  1. Initial State (maxCellSize)
    • The entire image is represented by only a few large cells.
    • Each cell represents a broad area’s average color using an emoji.
  2. Intermediate State
    • As the cell size decreases, more details emerge.
    • The key features of the image start to become clearer.
  3. Final State (minCellSize)
    • The smallest cell size is reached, showing the maximum level of detail.
    • The image’s features are most accurately represented.

2.2 Color Analysis and Emoji Mapping

Each cell is converted to an emoji through the following process:

  1. Area Averaging
    • Collects RGB values of all pixels within the cell.
    • Calculates the average to determine the representative color of the area.
  2. Color Space Conversion
    • Converts RGB to HSB (Hue, Saturation, Brightness).
    • Allows for analysis in a way that’s closer to human color perception.
  3. Emoji Selection
    • An emoji set is chosen based on the color (Hue).
    • A specific emoji is selected from the set based on the brightness.

2.3 Rectangular Area Processing (Recursive Tiling)

Challenges and solutions arising from changing cell sizes:

  1. Boundary Handling
    • Occurs when the image size does not divide evenly by the current cell size.
    • Results in rectangular areas without emojis on the edges.
  2. Recursive Solution
    • Divides rectangular areas into the largest possible squares and the remaining area.
    • Recursively applies the same process to the remaining area.
    • Continues until all space is covered with squares.
function setup() {
  createCanvas(400, 400);
  let currentCellSize = 330;
  for (let x = 0; x < width; x += currentCellSize) {
    for (let y = 0; y < height; y += currentCellSize) {
      // Calculate remaining space (boundary handling)
      // 남은 공간 계산 (경계 처리)
      let remainingWidth = min(currentCellSize, width - x);
      let remainingHeight = min(currentCellSize, height - y);

      // Divide the area into squares
      // 영역을 정사각형으로 분할
      divideIntoSquares(x, y, remainingWidth, remainingHeight);
    }
  }
  noLoop();
}

function drawSquareCell(x, y, size) {
  rect(x, y, size, size);
}

function divideIntoSquares(x, y, width, height) {
  // Draw immediately if it's a square
  // 정사각형인 경우 바로 그리기
  if (width === height) {
    drawSquareCell(x, y, width);
    return;
  }

  // Draw square based on the smaller dimension
  // 더 작은 변을 기준으로 정사각형 그리기
  let squareSize = min(width, height);

  if (width > height) {
    // If the width is greater than the height
    // 가로가 더 긴 경우
    drawSquareCell(x, y, squareSize);
    // Recursively handle the remaining area
    // 남은 영역을 재귀적으로 처리
    if (width - squareSize > 0) {
      divideIntoSquares(x + squareSize, y, width - squareSize, height);
    }
  } else {
    // If the height is greater than the width
    // 세로가 더 긴 경우
    drawSquareCell(x, y, squareSize);
    // Recursively handle the remaining area
    // 남은 영역을 재귀적으로 처리
    if (height - squareSize > 0) {
      divideIntoSquares(x, y + squareSize, width, height - squareSize);
    }
  }
}

Advantages

  • Complete coverage without empty spaces
  • Maintains square shape to ensure consistency of emoji expression
  • Creates a natural visual flow

2.4 Dual-Layer Visualization Strategy

Each square cell consists of two visual layers:

  1. Background Layer
    • Based on the current animation color (currentHue), with a semi-transparent background.
    • Maintains constant saturation (50) and brightness (70) in the HSB color space.
  2. Emoji Layer
    • The selected representative emoji based on pixel analysis.
    • Blends smoothly with the background using a globalAlpha of 0.4.
    • Adjusts dynamically to fit the cell size.

Advantages of this dual-layer approach:

  1. Visual Richness
    • Creates a composite visual effect beyond simple emoji layout.
    • Harmonizes gradients of background color with the emojis.
  2. Layered Information
    • Allows intuitive image reproduction with emojis.
    • Adds additional visual context through background color.
  3. Expression of Color Change
    • Adds a sense of change by contrasting the animated background color and emoji.
    • Creates contrast between static emojis and a dynamic background.

This approach enhances the visual experience by moving beyond a simple emoji transformation, offering a progressive detail representation and perfect spatial utilization.

3. Main Features

3.1 Emoji Mapping by Color

Reference: https://developer.mozilla.org/en-US/docs/Web/CSS/hue

const colorEmojis = {
  black: ['⚫️', '⬛️', '🖤', '📱', '💻', '⌚️', '🎮', '🎥'],
  gray: ['🌫️', '🐁', '🐘', '🦏', '🪨', '🗿', '🌪️', '🦭'],
  white: ['⚪️', '⬜️', '🤍', '🕊️', '🥚', '🌥️', '🍚', '🏐'],

  // Red tones (Hue 341-20 degrees)
  // 빨강계열 (341-20도)
  red: ['❤️', '🍎', '🍓', '🌹', '🏮', '🔴', '🫕', '🦞'],

  // Orange tones (Hue 21-45 degrees)
  // 주황계열 (21-45도)
  orange: ['🧡', '🍊', '🥕', '🦊', '🎃', '🦁', '🥭', '🦒'],

  // Yellow tones (Hue 46-70 degrees)
  // 노랑계열 (46-70도)
  yellow: ['💛', '🌟', '⭐️', '🌻', '🍋', '🐥', '🌼', '🌞'],

  // Yellow-green tones (Hue 71-110 degrees)
  // 연두계열 (71-110도)
  yellowGreen: ['🥬', '🥝', '🌿', '🍐', '🪴', '🌱', '🥒', '🐸'],

  // Green tones (Hue 111-170 degrees)
  // 초록계열 (111-170도)
  green: ['💚', '🌲', '🌳', '🥑', '🍀', '🌵', '🐢', '🥦'],

  // Cyan tones (Hue 171-210 degrees)
  // 청록/하늘계열 (171-210도)
  cyan: ['💠', '💧', '❄️', '🧊', '🩵', '🔹', '🎽', '🥏'],

  // Blue tones (Hue 211-260 degrees)
  // 파랑계열 (211-260도)
  blue: ['💙', '🌌', '🔷', '🫐', '👖', '🌀', '🥶', '🔵'],

  // Purple tones (Hue 261-290 degrees)
  // 보라계열 (261-290도)
  purple: ['💜', '🍇', '🔮', '☂️', '🌂', '👾️', '🟣', '🍆'],

  // Pink tones (Hue 291-340 degrees)
  // 분홍계열 (291-340도)
  pink: ['🎀', '🌸', '🍑', '🦩', '🧠', '🐷', '🌺', '👛']
};

Classification into 10 main color groups based on HSB color space:

  • Each color group is mapped to 8 related emojis.
  • Black, white, and gray are processed separately based on saturation and brightness.

3.2 Dynamic Cell Size Adjustment

function updateAnimationValues() {
  // Update cell size
  // 셀 크기 업데이트
  if (currentCellSize <= minCellSize) {
    cellSizeChangeRate = 0;  // Stop changing size if minimum is reached
  } 
  currentCellSize -= cellSizeChangeRate;

  // Update color change
  // 색상 변화 업데이트
  if (currentHue > 210) {
    hueChangeSpeed = -5; // Reverse direction if hue exceeds 210
  }
  else if (currentHue < 0) {
    hueChangeSpeed = 5; // Adjust direction if hue goes out of range (0-350)
  }
  currentHue += hueChangeSpeed;
}

Gradual change from maximum cell size (maxCellSize) to minimum cell size (minCellSize)

  • Implements animation to create a zoom-in/zoom-out effect.

3.3 Recursive Area Division

function draw() {
  background(255);

  updateAnimationValues();     // Update animation values
  // 애니메이션 값 업데이트

  sourceImage.loadPixels();    // Load image pixel data
  // 이미지 픽셀 데이터 로드

  // Loop through the image in cell units
  // 이미지를 셀 단위로 순회
  for (let x = 0; x < sourceImage.width; x += currentCellSize) {
    for (let y = 0; y < sourceImage.height; y += currentCellSize) {
      // Calculate remaining space (boundary handling)
      // 남은 공간 계산 (경계 처리)
      let remainingWidth = min(currentCellSize, sourceImage.width - x);
      let remainingHeight = min(currentCellSize, sourceImage.height - y);

      // Divide the area into squares
      // 영역을 정사각형으로 분할
      divideIntoSquares(x, y, remainingWidth, remainingHeight);
    }
  }
}

// Function to recursively divide rectangular areas into squares
// 직사각형 영역을 정사각형으로 재귀적으로 분할하는 함수
function divideIntoSquares(x, y, width, height) {
  // Draw immediately if it's a square
  // 정사각형인 경우 바로 그리기
  if (width === height) {
    drawSquareCell(x, y, width);
    return;
  }

  // Draw square based on the smaller dimension
  // 더 작은 변을 기준으로 정사각형 그리기
  let squareSize = min(width, height);

  if (width > height) {
    // If the width is greater than the height
    // 가로가 더 긴 경우
    drawSquareCell(x, y, squareSize);
    // Recursively handle the remaining area
    // 남은 영역을 재귀적으로 처리
    if (width - squareSize > 0) {
      divideIntoSquares(x + squareSize, y, width - squareSize, height);
    }
  } else {
    // If the height is greater than the width
    // 세로가 더 긴 경우
    drawSquareCell(x, y, squareSize);
    // Recursively handle the remaining area
    // 남은 영역을 재귀적으로 처리
    if (height - squareSize > 0) {
      divideIntoSquares(x, y + squareSize, width, height - squareSize);
    }
  }
}

Recursively divides rectangular areas into squares

  • Processes the entire canvas by dividing it into uniformly sized square cells.

3.4 Pixel Analysis and Emoji Selection

// Function to draw a square cell
// 정사각형 셀을 그리는 함수
function drawSquareCell(x, y, size) {
  colorMode(RGB);
  // Variables for calculating average color values
  // 평균값 계산을 위한 변수들
  let sumR = 0, sumG = 0, sumB = 0;
  let count = 0;

  // Sum up all pixel color values within the area
  // 해당 영역 내의 모든 픽셀 색상값 합산
  for (let dx = 0; dx < size; dx++) {
    for (let dy = 0; dy < size; dy++) {
      let p = (y + dy) * sourceImage.width + (x + dx);
      let i = p * 4;

      sumR += sourceImage.pixels[i];
      sumG += sourceImage.pixels[i + 1];
      sumB += sourceImage.pixels[i + 2];
      count++;
    }
  }

  // Calculate averages
  // 평균 계산
  let avgR = sumR / count;
  let avgG = sumG / count;
  let avgB = sumB / count;

  let c = [avgR, avgG, avgB, 255];
  let h = hue(c);
  let s = saturation(c);
  let br = brightness(c);

  // Select background color and emoji
  // 배경색과 이모지 선택
  let emojiSet;
  colorMode(HSB, 360, 100, 100, 1);

  if (s < 20 && br < 30) {
    emojiSet = colorEmojis.black;
  } else if (s < 20 && br >= 30 && br <= 70) {
    emojiSet = colorEmojis.gray;
  } else if (s < 20 && br > 70) {
    emojiSet = colorEmojis.white;
  } else {
    if (h <= 20 || h >= 340) {
      emojiSet = colorEmojis.red;
    }
    else if (h <= 45) {
      emojiSet = colorEmojis.orange;
    }
    else if (h <= 90) {
      emojiSet = colorEmojis.yellow;
    }
    else if (h <= 110) {
      emojiSet = colorEmojis.yellowGreen;
    }
    else if (h <= 170) {
      emojiSet = colorEmojis.green;
    }
    else if (h <= 210) {
      emojiSet = colorEmojis.cyan;
    }
    else if (h <= 260) {
      emojiSet = colorEmojis.blue;
    }
    else if (h <= 290) {
      emojiSet = colorEmojis.purple;
    }
    else {
      emojiSet = colorEmojis.pink;
    }
  }

  // Select emoji within the color set based on brightness
  // 밝기에 따라 같은 색상 세트 내에서 이모지 선택
  let emojiIndex = floor(map(br, 0, 100, 0, emojiSet.length));
  emojiIndex = constrain(emojiIndex, 0, emojiSet.length - 1);


  // Drawing background cells 배경셀 그리기
  noStroke();
  fill(currentHue, 50, 70); // hsb mode
  square(x, y, size);


  // Set emoji size to fit the cell
  // 이모지 크기를 셀 크기에 맞게 설정
  textAlign(CENTER, CENTER);
  textSize(size);
  push();
  drawingContext.globalAlpha = 0.4; // Value between 0.0 and 1.0
  text(emojiSet[emojiIndex], x + size / 2, y + size / 2 + size * 0.15);
  pop();
}

Calculates the average RGB value of each cell area

  • Converts to HSB color space to analyze hue, saturation, and brightness.
  • Selects the appropriate emoji based on the analysis.

4. Distinctive Implementation Points

  1. Transparency Effect
    • Uses drawingContext.globalAlpha to apply transparency to emojis.
    • Implements a smooth blending effect between background color and emojis.
  2. Dynamic Color Change
    • Continuously changes the currentHue value to create a background color animation.
    • Cycles color values between 0 and 360 degrees.
  3. Adaptive Cell Size
    • Calculates remainingWidth/Height to prevent cells from being cut off at the canvas boundaries.
    • Uses recursive tiling to fill all areas without gaps.