• bitcoinBitcoin(BTC)$61,964.00-0.89%
  • ethereumEthereum(ETH)$1,742.33-1.45%
  • tetherTether(USDT)$1.00-0.01%
  • binancecoinBNB(BNB)$573.45-0.97%
  • usd-coinUSDC(USDC)$1.000.00%
  • rippleXRP(XRP)$0.95-0.63%
  • solanaSolana(SOL)$72.50-0.85%
  • tronTRON(TRX)$0.308799-0.37%
  • Figure HelocFigure Heloc(FIGR_HELOC)$1.02-0.62%
  • HyperliquidHyperliquid(HYPE)$65.74-1.19%
  • zcashZcash(ZEC)$867.35-3.51%
  • moneroMonero(XMR)$494.45-1.16%
  • dogecoinDogecoin(DOGE)$0.057406-1.69%
  • leo-tokenLEO Token(LEO)$9.24-0.61%
  • whitebitWhiteBIT Coin(WBT)$60.04-0.95%
  • chainlinkChainlink(LINK)$8.87-1.40%
  • USDSUSDS(USDS)$1.00-0.01%
  • cardanoCardano(ADA)$0.149407-0.75%
  • Ethena USDeEthena USDe(USDE)$1.00-0.01%
  • stellarStellar(XLM)$0.132848-2.11%
  • daiDai(DAI)$1.00-0.01%
  • bitcoin-cashBitcoin Cash(BCH)$205.900.67%
  • nearNEAR Protocol(NEAR)$3.05-8.40%
  • litecoinLitecoin(LTC)$41.96-4.36%
  • CantonCanton(CC)$0.079337-2.77%
  • RainRain(RAIN)$0.004353-4.71%
  • avalanche-2Avalanche(AVAX)$6.90-1.15%
  • uniswapUniswap(UNI)$4.85-5.50%
  • the-open-networkGram (prev. Toncoin)(GRAM)$1.02-0.41%
  • suiSui(SUI)$0.70-1.94%
  • USD1USD1(USD1)$1.000.01%
  • tether-goldTether Gold(XAUT)$3,546.35-4.26%
  • Ripple USDRipple USD(RLUSD)$1.00-0.01%
  • Global DollarGlobal Dollar(USDG)$1.00-0.01%
  • paypal-usdPayPal USD(PYUSD)$1.00-0.02%
  • Ondo US Dollar YieldOndo US Dollar Yield(USDY)$1.150.04%
  • hedera-hashgraphHedera(HBAR)$0.053356-3.52%
  • BittensorBittensor(TAO)$188.28-1.38%
  • shiba-inuShiba Inu(SHIB)$0.000004-1.29%
  • crypto-com-chainCronos(CRO)$0.040894-2.19%
  • quant-networkQuant(QNT)$143.23-3.86%
  • Spiko Amundi Overnight Swap Fund (EUR)Spiko Amundi Overnight Swap Fund (EUR)(EURSAFO)$1.120.04%
  • Circle USYCCircle USYC(USYC)$1.140.01%
  • MemeCoreMemeCore(M)$0.84-6.84%
  • okbOKB(OKB)$84.55-2.96%
  • Falcon USDFalcon USD(USDF)$1.000.00%
  • United StablesUnited Stables(U)$1.000.03%
  • aaveAave(AAVE)$103.09-4.86%
  • pax-goldPAX Gold(PAXG)$3,545.54-4.27%
  • BlackRock USD Institutional Digital Liquidity FundBlackRock USD Institutional Digital Liquidity Fund(BUIDL)$1.000.00%
TradePoint.io
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop
No Result
View All Result
TradePoint.io
No Result
View All Result

Researchers Shanghai AI Lab and SenseTime Propose MM-Grounding-DINO: An Open and Comprehensive Pipeline for Unified Object Grounding and Detection

January 17, 2024
in AI & Technology
Reading Time: 6 mins read
A A
Researchers Shanghai AI Lab and SenseTime Propose MM-Grounding-DINO: An Open and Comprehensive Pipeline for Unified Object Grounding and Detection
ShareShareShareShareShare

Object detection plays a vital role in multi-modal understanding systems, where images are input into models to generate proposals aligned with text. This process is crucial for state-of-the-art models handling Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC). OVD models are trained on base categories in zero-shot scenarios but must predict both base and novel categories within a broad vocabulary. PG provides a phrase to describe candidate categories and output corresponding boxes, while REC accurately identifies a target from text and outlines its position using a bounding box. Grounding-DINO addresses OVD, PG, and REC, gaining widespread adoption for diverse applications. 

https://arxiv.org/abs/2401.02361v2

Researchers from Shanghai AI Lab and SenseTime Research have developed MM-Grounding-DINO, a user-friendly and open-source pipeline created using the MMDetection toolbox. It utilizes diverse vision datasets for pre-training and a range of detection and grounding datasets for fine-tuning. A comprehensive analysis of reported results and detailed settings for reproducibility are provided. Through extensive experiments on benchmarks, MM-Grounding-DINO-Tiny surpasses the performance of the Grounding-DINO-Tiny baseline. 

https://arxiv.org/abs/2401.02361v2

MM-Grounding-DINO builds upon the foundation of Grounding-DINO. It operates by aligning textual descriptions with corresponding generated bounding boxes in images with varied shapes. The main components of the MM-Grounding-DINO include a text backbone responsible for extracting features from text, an image backbone for extracting features from images, a feature enhancer for thorough fusion of image and text features, a language-guided query selection module for initializing queries, and a cross-modality decoder for refining bounding boxes.

When presented with an image-text pair, MM-Grounding-DINO employs an image backbone to extract features from the image at various scales. Simultaneously, a text backbone extracts features from the accompanying text. These extracted features are input into a feature enhancer module, facilitating cross-modality fusion. Within this module, text and image features undergo fusion through a Bi-Attention Block, encompassing text-to-image and image-to-text cross-attention layers. Subsequently, the fused features undergo further enhancement through vanilla self-attention and deformable self-attention layers, followed by a Feedforward Network (FFN) layer.

The study presents an open, comprehensive pipeline for unified object grounding and detection covering OVD, PG, and REC tasks. The model’s performance is evaluated through a visualization-based analysis, which reveals inaccuracies in the ground-truth annotations of the evaluation dataset. The MM-Grounding-DINO model achieves state-of-the-art performance in zero-shot settings on COCO, with a mean average precision (mAP) of 52.5. The MM-Grounding-DINO model also outperforms fine-tuned models in various domains, including marine objects, brain tumor detection, urban street scenes, and people in paintings, setting new benchmarks for mAP. 

https://arxiv.org/abs/2401.02361v2

In conclusion, The study introduces a comprehensive and open pipeline for unified object grounding and detection, addressing tasks like OVD, PG, and REC. The model exhibits notable improvements in mAP across various datasets, such as COCO and LVIS, through fine-tuning. The model’s predictions’ precision surpasses existing annotations for specific objects. The authors propose an extensive evaluation framework facilitating systematic assessment across diverse datasets, including COCO, LVIS, RefCOCOg, Flickr30k Entities, ODinW1335, and Description Detection Dataset (D3).


Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our Telegram Channel


YOU MAY ALSO LIKE

John Deere Buys Orchard AI to Pair Crop Scans with Farm Equipment – Unite.AI

Ericsson and Altair Test a Lower-Power 5G Link for AI Wearables – Unite.AI

Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.


🐝 Join the Fastest Growing AI Research Newsletter Read by Researchers from Google + NVIDIA + Meta + Stanford + MIT + Microsoft and many others…


Credit: Source link

ShareTweetSendSharePin

Related Posts

John Deere Buys Orchard AI to Pair Crop Scans with Farm Equipment – Unite.AI
AI & Technology

John Deere Buys Orchard AI to Pair Crop Scans with Farm Equipment – Unite.AI

November 20, 2026
Ericsson and Altair Test a Lower-Power 5G Link for AI Wearables – Unite.AI
AI & Technology

Ericsson and Altair Test a Lower-Power 5G Link for AI Wearables – Unite.AI

November 20, 2026
Job Posting Seemingly Confirms Apple Is Working On Its Own AI Agent For Coding And Other Tasks
AI & Technology

Job Posting Seemingly Confirms Apple Is Working On Its Own AI Agent For Coding And Other Tasks

November 20, 2026
The GTA 6 Star System Says A Lot More Than Just Your Wanted Level
AI & Technology

The GTA 6 Star System Says A Lot More Than Just Your Wanted Level

November 19, 2026
Next Post
Meet the Press NOW — Aug.11

Meet the Press NOW — Aug.11

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Search

No Result
View All Result
IS NVIDIA A GOOD INVESTMENT AFTER REPORTING EARNINGS?!

IS NVIDIA A GOOD INVESTMENT AFTER REPORTING EARNINGS?!

November 18, 2026
Debbie Shields speaks out on her viral moment riffing with Conan O’Brien

Debbie Shields speaks out on her viral moment riffing with Conan O’Brien

November 18, 2026
Good News: Retired custodian surprised with new school named in her honor

Good News: Retired custodian surprised with new school named in her honor

November 16, 2026

About

Learn more

Our Services

Legal

Privacy Policy

Terms of Use

Bloggers

Learn more

Article Links

Contact

Advertise

Ask us anything

©2020- TradePoint.io - All rights reserved!

Tradepoint.io, being just a publishing and technology platform, is not a registered broker-dealer or investment adviser. So we do not provide investment advice. Rather, brokerage services are provided to clients of Tradepoint.io by independent SEC-registered broker-dealers and members of FINRA/SIPC. Every form of investing carries some risk and past performance is not a guarantee of future results. “Tradepoint.io“, “Instant Investing” and “My Trading Tools” are registered trademarks of Apperbuild, LLC.

This website is operated by Apperbuild, LLC. We have no link to any brokerage firm and we do not provide investment advice. Every information and resource we provide is solely for the education of our readers. © 2020 Apperbuild, LLC. All rights reserved.

No Result
View All Result
  • Main
  • AI & Technology
  • Stock Charts
  • Market & News
  • Business
  • Finance Tips
  • Trade Tube
  • Blog
  • Shop

© 2023 - TradePoint.io - All Rights Reserved!