( 12 ) United States Patent

US010866985B2

(12) United States Patent

Chandra Sekar Rao et al.

(10) Patent No.: US 10,866,985 B2 (45) Date of Patent: Dec. 15, 2020

(54) IMAGE-BASED SEARCH AND RECOMMENDATION TECHNIQUES

IMPLEMENTED VIA ARTIFICIAL

INTELLIGENCE

(71 ) Applicant: EMC IP Holding Company LLC, Hopkinton, MA (US)

(72) Inventors: Venkata Chandra Sekar Rao, Bangalore (IN); Neeraj Tiwari, Bangalore (IN); Kalpana Razdan, Bangalore (IN); Sumit Gupta, Bangalore (IN)

(73) Assignee: EMC IP Holding Company LLC,

Hopkinton, MA (US)

( * ) Notice: Subject to any disclaimer, the term ofthis patent is extended or adjusted under 35 U.S.C. 154(b ) by 283 days.

(21 ) Appl. No.: 16/048,787

(22) Filed: Jul. 30, 2018

( 65 )

Prior Publication Data

US 2020/0034455 A1 Jan. 30, 2020

(51) Int. Cl.

G06Q 30/00

(2012.01 )

G06F 16/532

( 2019.01)

(Continued )

(52) U.S. CI.

CPC

G06F 16/532 (2019.01); GO6K 9/46

(2013.01); GO6K 9/6202 (2013.01); GOON 5/02 (2013.01); G06Q 30/0627 (2013.01)

(58) Field of Classification Search

CPC

G06Q 30 / 0601-0645

(Continued )

( 56 )

References Cited

U.S. PATENT DOCUMENTS

9,830,561 B2 11/2017 Pruthi et al . 10,043,109 B1 * 8/2018 Du

(Continued )

G06K 9/4652

OTHER PUBLICATIONS

Life: Online: 100 most useful websites: Cream of the crop: With internet activity at an all-time high, online asked specialist journal ists to select their favourite sites from across the web. (Dec. 16, 2004). The Guardian Retrieved from

professional/docview /246298150 ?accountid = 131444.*

(Continued )

Primary Examiner Resha Desai

(74) Attorney, Agent, or Firm -- Ryan, Mason & Lewis,

LLP

(57 )

ABSTRACT

Methods, apparatus, and processor-readable storage media for image-based search and recommendation techniques implemented via artificial intelligence are provided herein . An example computer-implemented method includes detect ing, in response to a user search query comprising an image, an object in the image by applying one or more artificial intelligence algorithms to the image; determining one or

more features of the object by applying the one or more

artificial intelligence algorithms to one or more portions of the image containing at least a portion of the object; iden tifying the detected object as a particular enterprise offering

based at least in part on the one or more determined features

ofthe object; determining one or more additional enterprise

offerings based at least in part on the identified enterprise offering; outputting, to the user, information pertaining to the identified enterprise offering and information pertaining

to the one or more additional enterprise offerings.

20 Claims, 10 Drawing Sheets

102-1

102?2

100

USER

USER

DEVICE DEVICE

- 102-K

USER DEVICE

NETWORK 104

_106

105

VISUAL SEARCH AND

RECOMMENDATION SYSTEM

s120

PROCESSOR

130

ARTIFICIAL INTELLIGENCE- (AI-)

BASED OBJECT DETECTOR

132

AI-BASED FEATURE EXTRACTOR

134

OFFERING MATCH IDENTIFIER

- 136

RECOMMENDATION GENERATOR

DATABASE

107

ENTERPRISE OFFERING

DATA

122

MEMORY 124

NETWORK INTERFACE

108

INPUT-OUTPUT DEVICES

US 10,866,985 B2

Page 2

(51) Int. Ci.

GOON 5/02

( 2006.01 )

G06K 9/62

( 2006.01 )

G06K 9/46

( 2006.01)

G06Q 30/06

( 2012.01 )

(58) Field of Classification Search

USPC

705 / 26.1?27.2

See application file for complete search history.

( 56 )

References Cited

U.S. PATENT DOCUMENTS

10,109,051 B1 * 10/2018 Natesh

10,346,893 B1 * 7/2019 Duan 2016/0224664 Al 8/2016 Noren et al . 2016/0378867 Al 12/2016 Panuganty 2017/0278135 A1 * 9/2017 Majumdar

G06K 9/4652 G06F 16/24578

G06K 9/00362

OTHER PUBLICATIONS

Wikipedia, Scale Invariant Feature Transform , .

org/w /index.php ?title =Scale -invariant_feature_transform & oldid = 847912019 , Jun . 28 , 2018 .

* cited by examiner

U.S. Patent Dec. 15, 2020 Sheet 1 of 10

US 10,866,985 B2

105

120 130 132 134 136

122 124

108

AI?()ANRTEILIFGCE-INACLE

RGECONMADTIR BODEATJSCEODTR A-BFEXTRSAICUETROD OMIDFAENTRCIHEG

RSECYOMNTDAEIVSAIENURACDLH PROCESR

MEORY NIETWRFOACKE I-ODEUNVTPICUETS

1-K02

USER DEVICE

106 107

102-2 USERDEVICE NETWORK DATBASE ENTRPISEOFERINGDATA

102-1 USER DEVICE

104 F.1IG 100

U.S. Patent Dec. 15 , 2020 Sheet 2 of 10

US 10,866,985 B2

208

VSIEAURCLH ENGINE

206

AROUTIER

F.2IG

204

210

ENTRPISE DATBSE

GA-ENRIATED OUTPUT

,202

UDESVIECRS

U.S. Patent Dec. 15, 2020 Sheet 3 of 10

US 10,866,985 B2

FEATURE VECTOR 0

1

1

0

1

0

1

310

1x0

4x 1096

CNOEVULRTIAON Hanes14x512 N(C)FUENTNCWTOINROKN308 $ 1x1 2x5182 7x5x27x5612

1x1228

4x 96 49x6

MaxPol MaxPol ) 47: (x47

2) 3( :x23

1x 956

8 x 96

CONV CONV 48) ( :x48

) ( 8: 4x84

5x7

2x9

SUBAMPLE SUBAMPLE( 4 : 1 )

( 8 : 1 )

2x36244

312

306

n-embeding CNN

304-2 304-3

F.3IG

HLIONSGE m( 0 , g + D q p ) - n aDx

p-embeding

CNN

304-1

q-embeding CNN

NEGATIVEI( n )MAGE

POSITVEI( P )MAGE ITRMAPLGET

302

QUERYI( 9 )MAGE

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download