By using this site, you agree to the Privacy Policy and Terms of Use.
Accept

News Junction

Notification Show More
Font ResizerAa
  • Home
  • World News
    World NewsShow More
    Saudis use burgers to impress Trump (VIDEO) — RT World News
    Saudis use burgers to impress Trump (VIDEO) — RT World News
    May 14, 2025
    Sony shares rise in choppy trade as PlayStation maker announces .7 billion share buyback
    Sony shares rise in choppy trade as PlayStation maker announces $1.7 billion share buyback
    May 14, 2025
    U.S. to cut tariffs on small packages from China to as low as 30% – National
    U.S. to cut tariffs on small packages from China to as low as 30% – National
    May 14, 2025
    A book points to Biden’s decline and Democrats’ cowardice: 6 takeaways
    A book points to Biden’s decline and Democrats’ cowardice: 6 takeaways
    May 14, 2025
    In one Indian city, reflective paint and bus stop sprinklers offer relief from killer heat
    In one Indian city, reflective paint and bus stop sprinklers offer relief from killer heat
    May 14, 2025
  • Business
    BusinessShow More
    Ukraine blows up bridges to consolidate its positions in Russia
    Ukraine blows up bridges to consolidate its positions in Russia
    August 18, 2024
    Commentary: AI phones from Google and Apple will erode trust in everything
    Commentary: AI phones from Google and Apple will erode trust in everything
    August 18, 2024
    The most famous Indian Dishes – Insights Success
    The most famous Indian Dishes – Insights Success
    August 18, 2024
    Life on the road as a female long rides cyclist
    Life on the road as a female long rides cyclist
    August 18, 2024
    UK inflation rises to 2.2%
    UK inflation rises to 2.2%
    August 18, 2024
  • Cryptocurrency
    CryptocurrencyShow More
    Top South Korean presidential hopefuls support legalizing Bitcoin ETFs
    Top South Korean presidential hopefuls support legalizing Bitcoin ETFs
    May 14, 2025
    Ex-SEC Chair Gary Gensler privately supported crypto — McHenry
    Ex-SEC Chair Gary Gensler privately supported crypto — McHenry
    May 14, 2025
    Bitcoin Pepe nears .2M on Solana hype
    Bitcoin Pepe nears $8.2M on Solana hype
    May 14, 2025
    eToro prices IPO at , eyes Nasdaq amid shifting sentiment
    eToro prices IPO at $52, eyes Nasdaq amid shifting sentiment
    May 14, 2025
    Bitcoin (BTC) Still Dominates the Market With 3,000 Price Tag, But Investors Are Rushing to Get in On Ruvi AI (RUVI) Thanks to 100% Early Bonus and 20,000% Price Prediction
    Bitcoin (BTC) Still Dominates the Market With $103,000 Price Tag, But Investors Are Rushing to Get in On Ruvi AI (RUVI) Thanks to 100% Early Bonus and 20,000% Price Prediction
    May 14, 2025
  • Technology
    TechnologyShow More
    How to Improve Your Spotify Recommendations
    How to Improve Your Spotify Recommendations
    August 18, 2024
    X says it’s closing operations in Brazil
    X says it’s closing operations in Brazil
    August 18, 2024
    Supermoon set to rise: Top tips for amateur photographers | Science & Tech News
    Supermoon set to rise: Top tips for amateur photographers | Science & Tech News
    August 18, 2024
    Scientists Want to See Videos of Your Cat for a New Study
    Scientists Want to See Videos of Your Cat for a New Study
    August 18, 2024
    OpenAI’s new voice mode let me talk with my phone, not to it
    OpenAI’s new voice mode let me talk with my phone, not to it
    August 18, 2024
  • Entertainment
  • Sports News
  • People
  • Trend
Reading: Google unveils Lumiere generative AI to create more realistic images and videos from text
Share
Font ResizerAa

News Junction

  • World News
  • Business
  • Technology
  • Cryptocurrency
  • Trend
  • Entertainment
Search
  • Recent Headlines in Entertainment, World News, and Cryptocurrency – NewsJunction
  • World News
  • Business
  • Cryptocurrency
  • Technology
  • Entertainment
  • Sports News
  • People
  • Trend
Have an existing account? Sign In
Follow US
News Junction > Blog > Trend > Google unveils Lumiere generative AI to create more realistic images and videos from text
Google unveils Lumiere generative AI to create more realistic images and videos from text
Trend

Google unveils Lumiere generative AI to create more realistic images and videos from text

Published January 31, 2024
Share
5 Min Read
SHARE

Google unveils Lumiere – the latest in generative AI that creates realistic video clips from text. (Source: Google Research)

Google has unveiled Lumiere – the latest in realistic text-to-image and text-to-video generation using machine learning. A key innovation is the ability to create realistic motion such as walking that current generative AIs have trouble with. The software does this by creating all video frames at once rather than using keyframes and training to learn how moving objects should appear.

Google has unveiled Lumiere, the state-of-the-art in realistic text-to-image and video generative AI. The software greatly improves upon the motion by using a novel approach to video frame generation that creates all the frames in one pass to mitigate motion errors.

Generative image AI creates images from text. One key enabling this is the huge amount of online images and videos available for training. Another is the development of methods to associate all words in a language with each other through vectors. Therefore, AI can understand as a pair of words, or in a sentence, “I am” is more likely than “I unilaterally”. Image creation AI such as Stable Diffusion associates words with object images. Such AI understands the words “royal residence” are more closely associated with a “castle” image than a “house” image.

Generative video AI extends image AI to create videos from text. Lumiere competitors first create keyframes, then the frames in between. This is like a master animator drawing the beginning and end images of a basketball shot, then having an assistant draw the images in between. The issue is that motion errors often occur because the in-between images aren’t drawn correctly, so Lumiere bypasses this by creating all video frames without keyframing. Also, Lumiere is trained to know what moving objects look like at various image sizes, so its videos look superior.

Technically, Lumiere utilizes diffusion probabilistic models to generate images coupled with a Space-Time U-Net, a U-net architecture with temporal up and down scaling plus attention blocks added to the usual image resolution scaling. Down-scaling temporally simultaneously with resolution significantly reduces computational workloads while up-scaling coupled with a temporally-aware, spatial super-resolution model generates the high-resolution output. Still, image frame segmentation is required due to memory limitations, so Multidiffusion is used across overlapping, frame segment boundaries to help mitigate temporal motion artifacts.

Lumiere can be coupled with other AI to create a broader range of output. This includes:

  • Cinemagraphs – one section of an image is animated
  • Inpainting – one object in a video is replaced by another
  • Stylized generation – the appearance is re-created in another art style
  • Image-to-video – a desired image is animated
  • Video-to-video – videos are re-created in another art style

The video length is limited to 5 seconds while the ability to create video transitions and multiple camera angles are non-existent. Readers interested in experimenting with generative AI on their desktop computers should upgrade to a powerful video card (like this at Amazon) for the best performance during training.

Lumiere can create images and videos from text, stylized to match another art, and even replace objects. (Source: Google Research)
Lumiere can create images and videos from text, stylized to match another art, and even replace objects. (Source: Google Research)
Lumiere can animate a part of an image and the output can be fed into other AI easily. (Source: Google Research)
Lumiere can animate a part of an image and the output can be fed into other AI easily. (Source: Google Research)

David Chien
David Chien – News Writer – 78 articles published on Notebookcheck since 2023

Having worked at Activision, UCLA, Anime Expo and more, I’ve seen technology being used to save lives, create games, and create fantastic 3D VR/AR worlds. There’s always something fun in emerging technology that I want to get my hands on and all my friends turn to me to find the best for their needs, so I’m glad to bring my experience to Notebookcheck.

Please share our article, every link counts!

#Google #unveils #Lumiere #generative #create #realistic #images #videos #text

- Advertisement -
TAGGED:attention blocksbenchmarkscreatediffusiondiffusion probabilistic modelgenerativeGenerative AIgenerative videoGoogleGoogle Researchgraphics cardimage diffusionimageslaptopLumiereMultiDiffusionnetbooknotebookprocessorRealisticreportsreviewreviewsSpace-Time U-Netspatial super-resolutionspatial super-resolution modelStable Diffusiontemporaltemporal scalingtestteststexttext-to-imagetext-to-videoU-netunveilsvideos
Share This Article
Facebook Twitter Pinterest Whatsapp Whatsapp LinkedIn Email Copy Link Print
Share
What do you think?
Love0
Sad0
Happy0
Sleepy0
Angry0
Dead0
Wink0
Previous Article Microsoft, Alphabet under pressure to show AI earnings boost Microsoft, Alphabet under pressure to show AI earnings boost
Next Article Gisele Bündchen pays tribute to her late mother: “You were an angel on earth” Gisele Bündchen pays tribute to her late mother: “You were an angel on earth”
- Advertisement -

Latest Post

Top South Korean presidential hopefuls support legalizing Bitcoin ETFs
Top South Korean presidential hopefuls support legalizing Bitcoin ETFs
Cryptocurrency
Saudis use burgers to impress Trump (VIDEO) — RT World News
Saudis use burgers to impress Trump (VIDEO) — RT World News
World News
Ex-SEC Chair Gary Gensler privately supported crypto — McHenry
Ex-SEC Chair Gary Gensler privately supported crypto — McHenry
Cryptocurrency
Sony shares rise in choppy trade as PlayStation maker announces .7 billion share buyback
Sony shares rise in choppy trade as PlayStation maker announces $1.7 billion share buyback
World News
Bitcoin Pepe nears .2M on Solana hype
Bitcoin Pepe nears $8.2M on Solana hype
Cryptocurrency
eToro prices IPO at , eyes Nasdaq amid shifting sentiment
eToro prices IPO at $52, eyes Nasdaq amid shifting sentiment
Cryptocurrency
- Advertisement -

You Might Also Like

Top 100 Marketing Trends in August
Trend

Top 100 Marketing Trends in August

August 13, 2023
Inside an Election Denial Facebook Group on Primary Day
Trend

Inside an Election Denial Facebook Group on Primary Day

January 28, 2024
Commentary: Google ruled an illegal monopoly – where does the tech giant go from here?
Business

Commentary: Google ruled an illegal monopoly – where does the tech giant go from here?

August 17, 2024
Backbone One x Death Stranding inbound with free copy of the game
Trend

Backbone One x Death Stranding inbound with free copy of the game

January 28, 2024

About Us

NEWS JUNCTION (NewsJunction.xyz) Your trusted destination for global news. Stay informed with our timely and accurate reporting on diverse topics, including politics, technology, science, entertainment, sports, and more. Count on us for unbiased and reliable updates at your fingertips.

Quick Link

  • About
  • Disclaimer
  • Privacy Policy
  • Terms of Use
  • Contact

Top Categories

  • World News
  • Business
  • Technology
  • Entertainment
  • Cryptocurrency
  • Sports News
  • Trend
  • People

Subscribe

Subscribe to our newsletter to get our newest articles instantly!

    © 2023 News Junction.
    • Blog
    • Advertise
    • Contact
    Welcome Back!

    Sign in to your account

    Username or Email Address
    Password

    Lost your password?